Digital Asset Management

Extension Key: dam

Copyright 2003-2007, René Fritz <r.fritz@colorcube.de>

This document is published under the Open Content License

available from http://www.opencontent.org/opl.shtml

The content of this document is related to TYPO3

- a GNU/GPL CMS/Framework available from www.typo3.com

Table of Contents

Digital Asset Management	1

Introduction	1

News first	1

What is a Digital Asset Management System?	2

TYPO3 DAM	3

More about DAM and TYPO3	4

Beyond file handling	4

Applications	5

Outlook	5

Users manual	5

Backend modules	5

Element browser	8

Concepts	9

Selections	9

File usage as reference	9

Administration	10

Upgrade to TYPO3 4.1	10

Media SysFolder	10

General access	11

Access restrictions	11

Multi language support	11

TSconfig	12

DAM for Developers	13

Demo extension	13

Development	13

Planned features	14

Known problems	15

Changelog	16

Introduction

A DAM system is a tool to handle digital content like images, text files and virtually any other data format ... and so on.

But first of all ...

News first

This is the 1.0 release and includes lot's of changes and new features.

New in this release:

Multilanguage support

The media records can be created and edited in multiples languages.

File module

The Media>File>List module provide access to the file system while supporting the DAM functionality. This is a full replacement for the original file module.

Detection of file changes

Using file checksums allows the detection of externally renamed file or other unexpected file changes.

Silent indexing

When a new file occurs in the File module it will be indexed silently and is available in the DAM instantly.

Auto indexing

Index setups can be stored in folders (currently by using the dam_cron extension) to define the auto/silent indexing.

Improved meta services

The meta extract services are improved and are able to handle character encodings like UTF-8.

Selection trees for indexing time and file status

Two new selection trees allows the selection of files from the current status and the last indexing times.

Actions

Actions in the list modules like “delete” or “edit” can be extended by own extensions.

Previewer

“Previewer” can be added by own extensions. Previewer are components that shows the file content like a thumbnail. Previewer are used in the top header when editing a meta data record but will be used at more places later. (Hint: mouse over on the header image or editing a mp3 file).

Improved GUI

General GUI improvements and cleanups. Icons are skinnable and all texts can be translated.

Media>Tools module

This module provide administration functions.

Update status module

With the Tools>Update status module the whole index can easily be checked.

Cronjob script and module for configuration

A module for setting up an indexing configuration and a script

Configurable via Tsconfig

First TSconfig options are available for configuration.

Clean API

A clean documented API is available.

Still there are a few todo's that didn't make it into this release.

Todo's:

“Move file” function and similar

finish tx_dam_media API

Clipboard

This features will be delivered as soon as possible.

TYPO3 3.8

Support for TYPO3 V 3.8 is highly experimental and relies on some XCLASSes. I recommend to upgrade to TYPO3 V 4 instead of using that combination!!!

What is a Digital Asset Management System?

A DAM system is a tool to handle digital content like images, text files and virtually any other data format.

These media files can be attributed with meta information. Such information can describe the content (description, keywords, location), provide technical data (image size) or copyright information for example.

All these information can be used to handle, find and categorize the media files.

Two different forces drive the necessity for advanced management of such assets:

Large number: even smaller sites tend to produce large numbers of assets, often stored in random folders and with incoherent naming, making the organization and overview increasingly hard over time. Digital asset management adds a meta-layer of information to every file in a semi-automatic indexing process, allowing users to search for attributes in a database associated with the files.

Automated retrieval: assets as building blocks for content often are used to produce automated output, like image galleries and download areas. Digital asset management serves these output functions by providing an interface for retrieving assets filtered by any combination of meta data criteria.

TYPO3 DAM

The Digital Asset Management for TYPO3 was developed to address the growing demand for professional asset management in TYPO3 applications and implementations. Even though the first final version is still to come, previous versions have been used in various production settings.

The DAM is similar to the File module integrated into the TYPO3 backend as a main module. It will replace the File module when finished, which means all file handling like copying, deletion and renaming of files is provided by the DAM. Nevertheless the DAM don't use a new storage type for files. The files are stored into the file system as before.

More about DAM and TYPO3

Beyond file handling

Simple file handling can be improved when meta data is available, moreover the database based storage of meta data allows new features. As example we want to show assets can be used as reference.

TYPO3 regular technique to handle file content is to create a copy for each content element (record) where the file is inserted. This is a safe solution and there's no problem removing the original file. But this is also very limited when an editor wants to know where assets are in use on the website. Using references instead of copies gives more control over the available asset.

The usage of assets can be displayed when used by reference.

The DAM has experimental support for references which can be enabled with the extension manager. The submodule Media>Info>References displays a list of used assets. While the assets itself is listed it can be seen to which page and to which record the asset is linked to.

A warning will be given when a referenced assets should be deleted.

Deleting an assets which is in use as reference gives a warning and show the records where the assets is in use.

Applications

As stated in the beginning, meta data and their handling by the DAM serve many purposes associated with automatic retrieval of assets by criteria matching attributes of assets. Classic examples are image galleries and download areas, but the next step are product data management systems (PDM), like online product catalogues, e-commerce shops and the complete range of media management for print and other channels. One more hot topic is the integration with document management systems. As of today examples of almost all of these types of applications already exist and further development leading to standard DAM modules for these purposes are to surface in the mid-term.

Outlook

The future of the DAM, given the rather complete list of features already or close to publication, might rather lie with the aspirations of it's parts than with the ongoing development of the whole. The core DAM will be still refined and grow through extensions built on it's API, but the most thrilling prospects are offered by applying innovations and principles introduced to TYPO3 by the DAM to other field of application. Examples might be a coherent global meta data management no longer limited to assets but stretching to generic content generated in the CMS and layered from meta data used in authoring workflows to formats catering to public display and information retrieval by external systems, such as semantic browsers.

Users manual

Backend modules

The DAM consist of a main module 'Media' with submodules. The list of submodules can be varying depending of which additional DAM extensions are installed.

The main DAM modules are:

Media>File

This Module allows for a standard file view enhanced with functions for uploading files to the server, as well as the indexing submodule, which is used for creating meta data for batches or individual files.

The DAM counterpart of the old file module gives the user file manager feeling but with enhanced functionality.

Media>List

This module is used for handling files through the meta data layer, which have already been indexed previously. Files can be searched for and filtered by categories or other meta data.

The media files can be found by filters. Such a filter rule can be: "Show all files of the category 'Press' which are text type files but not PDF". This can be done easily by clicking on the + = - icons in the navigation frame. The selection result can be saved for later use.

Features:

searching by rules

saved selections (rules) for later use

editing of meta data

The DAM provides several ways to access and view your files.

Special features: Own category trees can be supplied by extensions. If one needs more or custom categorizations than the standard categories, extensions can be employed to enable this. Custom category trees show up in the navigation frame below the standard category tree by default.

Media>List>List

This submodule gives a compact list view on the current selection

Media>List>Thumbnail

Similar to the Media>List>List submodule this module displays the same output with larger thumbnails.

Media>List>Selection

The current selection can be viewed and modified here.

Here you can see that rules are used to create a selection. From the selection you can remove single files.

Media>List>Process

A current selection can be batch processed with this module. This can be used to enrich already indexed files with new meta data.

Features:

batch processing of the current selection

overwrite meta data of your selected files

append meta data: eg. add a keyword or add a category to your files

Batch processing fields can be extended by own extension. Special features include automatic meta data extract (EXIF, IPTC ... XMP), text extraction and automatic detection of content language.

Media>Indexing

The indexing module is used to index a bunch of files or folders with subfolders (in contrary to the file module which will index single files after upload).

Features:

mass indexing of files and folder

apply rules while indexing (eg. assign category from folder name)

preset meta data before indexing

set preset meta data not to be overwritten while indexing

save index setup as preset

The indexing module has four steps where the preset for the indexing will be done.

Special feature: Indexing rules can be extended by custom extensions. For example file names could have a special format which makes it easy to categorize the files. This can be processed by a custom indexing rule to match the file name pattern to categories during the indexing process.

Media>Categories

This is a temporary module to provide global category management and will be integrated seamlessly into the other modules at a later time

Media>Info

This submodule serves as the interface of the DAM for standard content authors, using assets already indexed for content creation and delivery.

Currently this submodule provides the following materials:

Media>Info>References

The references submodule gives an overview of the used assets in the system. This feature depends on the experimental references functionality.

Media>Tools

Generally Functions for Administrators are placed here.

Media>Tools>File check

This module is experimental and gives an overview about the indexed and non-indexed files.

Media>Tools>Cron Job

This module is similar to the indexing module with the difference that the indexing setup can be saved to be used as a preset for a cron script. The cron script itself can be called periodically or triggered by some event to index new files automatically while operating in the background.

Element browser

The element browser is TYPO3's ubiquitous tool enabling content authors to access files and assets from the webserver. The Element browser enhancements coming with the DAM include an interface for choosing files from categories or mime types. An integrated lightbox function is also planned and in production.

The DAM element browser integration gives access to the assets by categories and meta data.

Concepts

Selections

Selections are a main concept used to provide a powerful tool for retrieval of the stored media meta data. With selections the user can query the database with a visual tool to define the rules to search for media items.

Selection buttons and relation to the resulting selection rule. Clicking on the title and the + = - icons will combine a selection which helps to find the wanted media. The selection can be stored and used by external applications like Frontend plugins.

Any extension can extend the DAM with own selections. An example can be found in the dam_demo extension.

File usage as reference

TYPO3 regular technique to handle file content is to create a copy for each content element (record) where the file is inserted. This is a safe solution and there's no problem removing the original file. But this is also very limited when an editor wants to know where assets are in use on the website. Using references instead of copies gives more control over the available asset.

The usage of assets can be displayed when used by reference.

The DAM has experimental support for references which can be enabled with the extension manager. The submodule Media>Info>References displays a list of used assets. While the assets itself is listed it can be seen to which page and to which record the asset is linked to.

To enable this feature has to be activated in the Extension Manager. Go into the details view of the dam extension and activate the corresponding option.

Administration

Upgrade to TYPO3 4.1

TYPO3 4.1 has support for foreign MM table handling included. The extension mmforeign provided the functionality needed by extension using DAM file references like dam_ttcontent does.

mmforeign is no longer needed when TYPO3 4.1 is used. However the mm table format changed a little bit. The mm table use the field sorting_foreign which was sorting before. This means by upgrading to TYPO3 4.1 all sorting of file references in content elements would be lost.

DAM has an upgrade function builtin. Just go into the Extension Manager details view of the dam extension. That's where some configuration options are displayed. The conversion of the mm table for references will be done silently. This work as long as no editing of files references was done after upgrade.

Downgrade to 3.x-4.0 is not supported. To do so the field tx_dam_mm_ref.sorting_foreign needs to be renamed back to tx_dam_mm_ref.sorting.

Media SysFolder

The DAM creates a SysFolder 'Media' in the page tree by itself. This folder holds all DAM records.

To give users access to the Media items/records you have to add the Media SysFolder to their DB Mounts or you have to move the SysFolder to a location inside a DB Mount.

The records can be listed and edited with the normal List module. This is not recommended and should be accessed by administrators only. See section “Set Media SysFolder invisible” how to hide the SysFolder.

General access

Like other modules you have to grant access to the media modules. This can be done in the backend users record or the records of the usergroup the user is a member of.

In the usergroup record you can define with the exclude fields to which fields of the DAM record the usergroup shouldn't have access to.

Access restrictions

In user and usergroup records mountpoints similar to file- or db-mounts can be defined. Those mointpoints are define the access to parts of the selection trees. Some trees do not allow to mount parts but the whole tree only.

Further access related configurations can be done with TSconfig.

Multi language support

The DAM uses the system language support , which means the meta data records can be translated with so called overlay records. The translated records do not allow to edit data which is non-editable anyway or don't make sense to be translated. The behavior of each field can be reconfigured with an own extension by modifying TCA of course.

Please have a look in the TYPO3 documentation for a detailed description of the language support.

To activate the possibility to create and edit translated meta data, a translation of the Media SysFolder have to be created. This can be done with the Web>Page module for example. This is like creating a translation for a normal page.

In the Media>List module you can switch display of languages with a select box in the upper right corner. This will be improve to show records which are not yet translated.

When editing a meta data record it can be switched to other languages and new translations can be created.

TSconfig

Configuration of the DAM and it's module are provided through TSconfig. For User and Page TSconfig the same properties are available for configuration. Please keep in mind Page TSconfig will be read from the DAM SysFolder only.

Configuration is placed in

mod.txdamM1 ... module configuration

tx_dam. ... general DAM configuration

Disable submodules

Submodules can be disabled for a user (or globally) with this TSconfig:

mod.txdamM1_list.menu.function.tx_dam_list_list = 0

mod.txdamM1_list.menu.function.tx_dam_list_thumbs = 0

mod.txdamM1_list.menu.function.tx_dam_list_editsel = 0

mod.txdamM1_list.menu.function.tx_dam_list_batch = 0

mod.txdamM1_file.menu.function.tx_dam_file_list = 0

mod.txdamM1_file.menu.function.tx_dam_file_upload = 0

mod.txdamM1_file.menu.function.tx_damindex_index = 0

mod.txdamM1_index.menu.function.tx_damindex_index = 0

mod.txdamM1_info.menu.function.tx_daminfo_reference = 0

mod.txdamM1_tools.menu.function.tx_dam_tools_indexupdate = 0

mod.txdamM1_tools.menu.function.tx_dam_tools_filerelcheck = 0

mod.txdamM1_tools.menu.function.tx_damcron_modfunc1 = 0

Set Media SysFolder invisible

Media records in the Media SysFolder are not meant to be edited with another module than the DAM. While the SysFolder is visible to users, one can edit records with the Web>List module. With the following User TSconfig option the Media SysFolder (example page id: 23) can be made invisible in TYPO3 V4+.

options.hideRecords.pages = 23

(Note: in TYPO3 V 4.0.0 this works only when the SysFolder is not a DB mount itself but inside a DB Mount. As workaround move the Media SysFolder to a location inside a DB Mount)

Selections

The selection trees can be configured as follows:

// disable +=- icons by default

tx_dam.selections.default.disableModeSelIcons = 1

// disable txdamIndexRun selection tree (hidden)

tx_dam.selections.txdamIndexRun.disable = 1

// not disable +=- icons for txdamStatus selection tree

tx_dam.selections.txdamStatus.disableModeSelIcons = 0

// limit category selection depth to 0 (non-recursive)

tx_dam.selections.txdamCat.sublevelDepth = 0

Auto indexing

Auto indexing can be disabled by:

// disable auto indexing

tx_dam.indexing.auto.disable = 1

DAM for Developers

The DAM has not yet an API to access the functionality of the system which is of course the basis to develop applications based on the DAM. A first version of the API is planned to be shipped with the Beta 3 (end of Sept. 2005).

Nevertheless there are already (frontend) applications in production and in use that access the database tables of the DAM directly. The tables are not expected to be changed very much in the future, so that's currently the way to go till the API is available.

Also many paradigms introduced to TYPO3 through the DAM have found their way into other areas of development. One such example are services, that extend the DAM, but meanwhile are used to extend other parts of TYPO3 in a similar fashion as well. In that sense the influence of the DAM goes well beyond it's immediate use and is about to serve as a basis to many new functions (like a new ecommerce shop) and also brings new technical solutions to detail problems (like services) as well as introducing new backend interfaces (category trees, multiple step data handling).

Demo extension

To demonstrate the possibilities to extend the DAM with new features a demo extension is provided. While there's no further documentation for developers available at this time, the demo extension (dam_demo) is currently the best resource for developers.

Development

The project page on TYPO3.org gives further information about the planned features and the roadmap.

The mailing list is the place for discussions, sharing thought and ideas but also for organizing the further development.

Snapshot dowloads

http://typo3lab.colorcube.de/dam.html

Project page on TYPO3.org

http://typo3.org/development/projects/digital-asset-management/

Mailing list

http://lists.netfielders.de/cgi-bin/mailman/listinfo/typo3-project-dam/

Bugtracker

The bugtracker on http://bugs.typo3.org provides the category tx_dam for bug reports depending the DAM.

Planned features

Media>List>Lightbox

The lightbox module you can save and share your selections with. Selections can be stored in different folders. Folders can be shared with other users.

This way an editor can search images for an article and can hand over this selection to another editor which will write the article. In the element browser the lightbox items are available.

Of course a user can use that module to save and organize his selections for long use or just as notepad.

The following screenshot shows a first draft.

Media>File>Image processing

This is a small image processing module.

Features:

visuall cropping with mouse clicks

rescaling

filtering: brightness, contrast, ...

Secure File Management

In a further development step the DAM should be able to store its original files outside the document root of the webserver. Often the files themselves contain sensible material which must be protected from accidental direct URI-call in the status bar of the browser.

Managing file versions

Update a file with a new one with changed content while archiving older versions of a file.

Provide file variants

Eg. Images in different file versions (JPEG, CMYK-TIFF, ...) are managed with the same meta data record.

Grouping of elements

Eg. Press release: 1 PDF, 1 RTF and 3 images

This can be implemented in two ways:

a file entry that is also the parent for other files (files related to this one)

a pure container that groups files

Non-File Assets

Assets don't have to be files. Which means to support eg. content elements to be managed by the DAM.

New Image form element

To enable the possibilities of the DAM a new form element is planned. It allows to add images and other elements (caption, alt text, ...) dynamically. You can add for example a separate caption for each image and it's possible to fetch the caption content from the DAM.

Have a look at the faked screenshot for inspiration.

Features:

comfortable display and handling of the images

elements like caption and alt texts can be added to each image separately

meta data from the DAM can be inserted

each image can be added as copy or as reference

Known problems

none important

Changelog

V 1.0.x

Smaller bug fixes. See file dam/ChangeLog for details.

V 1.0

totally overhauled extension and refactored code – the basis for further development

Digital Asset Management - 16

Digital Asset Management

Kasper Skårhøj

2002-11-01T00:32:00

Kasper Skårhøj