What I learned as a Subject Matter Expert while creating my product
- Tags
- Definitions, Acronyms, Abbreviations
- Overview
- Multimedia IR systems
- Metadata
- Examples of incorrect search results
- Multimedia Organizers
- Integration
- Integration with digiKam
- Integration with Google Photos
- References
Multimedia IR Systems, Metadata, Multimedia Organizers, digiKam, Google Photos, Multimedia Data, Photo Album.
| # | Abbreviation or Acronym | Definition |
|---|---|---|
| 1 | digiKam | digiKam is an advanced open-source digital photo management application that runs on Linux, Windows, and MacOS. The application provides a comprehensive set of tools for importing, managing, editing, and sharing photos and raw files. |
| 2 | Exif | Exchangeable image file format (officially Exif, according to JEIDA/JEITA/CIPA specifications) is a standard that specifies the formats for images, sound, and ancillary tags used by digital cameras (including smartphones), scanners and other systems handling image and sound files recorded by digital cameras. |
| 3 | exiftool | ExifTool is a free and open-source software program for reading, writing, and manipulating image, audio, video, and PDF metadata |
| 4 | Google LLC is an American multinational technology company that specializes in Internet-related services and products, which include online advertising technologies, a search engine, cloud computing, software, and hardware. | |
| 5 | Google Photos | Google Photos is a photo sharing and storage service developed by Google. The service automatically analyzes photos, identifying various visual features and subjects. Users can search for anything in photos, with the service returning results from three major categories: People, Places, and Things. |
| 6 | IPTC | The International Press Telecommunications Council (IPTC), based in London, United Kingdom, is a consortium of the world's major news agencies, other news providers and news industry vendors and acts as the global standards body of the news media. |
| 7 | Multimedia data | Multimedia in principle means data of more than one medium. It usually refers to data representing multiple types of medium to capture information and experiences related to objects and events. Commonly used forms of data are numbers, alphanumeric, text, images, audio, and video. In common usage, people refer a data set as multimedia only when time-dependent data such as audio and video are involved. |
| 8 | Multimedia IR | Multimedia Information Retrieval (MIR) is an organic system made up of Text Retrieval (TR); Visual Retrieval (VR); Video Retrieval (VDR); and Audio Retrieval (AR) systems. So that each type of digital document may be analyzed and searched by the elements of language appropriate to its nature, search criteria must be extended. |
| 9 | metadata | Metadata is "data that provides information about other data". In other words, it is "data about data." Many distinct types of metadata exist, including descriptive metadata, structural metadata, administrative metadata, reference metadata and statistical metadata. |
| 10 | UI | is the space where interactions between humans and machines occur. The goal of this interaction is to allow effective operation and control of the machine from the human end, whilst the machine simultaneously feeds back information that aids the operators' decision-making process |
| 11 | XMP | The Extensible Metadata Platform (XMP) is an ISO standard, originally created by Adobe Systems Inc., for the creation, processing and interchange of standardized and custom metadata for digital documents and data sets. |
This article discusses the author's experience with Multimedia IR systems and the importance of metadata in maximizing the efficiency of multimedia data search. The author describes their analysis and comparison of various Multimedia IR systems before selecting the most suitable one. The article emphasizes the significance of metadata as a primary information source used by these systems for search following different criteria. The article provides examples of incorrect search results caused by inconsistent or non-existent metadata. The author also highlights the role of Multimedia Organizers and their integration with Multimedia IR systems to provide automation for importing new photo sets into existing photo albums. Finally, the article discusses the FVA Tool Set's integration with offline Multimedia IR systems through digiKam and online Multimedia IR systems through Google Photos.
My first step in the role of an SME was getting to know what Multimedia Data is. I started learning it as I understood that my whole photo album is Multimedia Data itself. Then I realised that I need to use a solution to search for Multimedia Data - Multimedia IR systems. However, there are many Multimedia IR systems, and it was a long way to select one of them to start searching more efficiently. Thus, several Multimedia IR systems, that are present on the market and available for end users, were analyzed.
I described the process of analysis and comparison for Multimedia IR systems here. Consequently, I was able to choose the most suitable system for me.
A short usage of these systems gave me an understanding that Multimedia IR systems significantly rely on the metadata as a main information source that is used for a search following different criteria, like device, time, location, etc. Therefore, the presence and consistency of metadata are very important for Multimedia IR systems to operate, as well as for other purposes, please see Three Cheers for Embedded Metadata for more details. Working experience with one of those systems shows that search results might be incomplete or even incorrect due to inconsistent or not-existing metadata. Metadata is usually kept inside each file as a header with a predefined format. There are plenty of metadata formats like Exif, IPTC or XMP.
The reasons for incorrect and absent metadata are quite different - from not setting proper data on the device that takes photos to privacy issues. This resource describes metadata importance in more detail and there is information about other reasons for incorrect and absent metadata.
Let's consider two examples of incorrect search results.
In the first case, we tried to search for photos with existing Geo positions and to see them on a map.
A search returned four photos taken near Ghana, marked by a red circle.
In fact, nobody took photos near Ghana.
In the second case, we searched for photos taken in 2011.
We got one video that was recognized as March 2020, underlined in red.
The system has provided such results due to incorrect metadata.
That being the case, I understood that Multimedia IR systems are not so effective without other tools, often called Multimedia Organizers. Usually, Multimedia Organizers provide automation for importing a new photo set into existing photo albums and it is not an implemented feature in Multimedia IR systems. In addition, they might identify and address some issues that shall be fixed before a user starts searching data using the Multimedia IR systems.
Then I started studying Multimedia IR systems the FVA Tool Set shall integrate with.
Integration with offline Multimedia IR systems is implemented through integration with digiKam, and integration with online Multimedia IR systems is implemented through integration with Google Photos.
As Multimedia IR systems perform a search by certain criteria, based on multimedia metadata, the FVA Software shall be able to work with the same multimedia metadata to be integrated with Multimedia IR systems.
For digiKam, I found that the Collections setting shall be used to define how it is configured to search through multimedia data. Monitor the albums for external changes (requires restart) setting shall be enabled. Taking into account this setting, digiKam will pick up external changes in the collection as soon as FVA Tool Set puts it in.
However, from the FVA Tool Set side, it is required to provide a possibility to configure storage in a way to be able to put multimedia data processed and to be integrated with digiKam. Therefore, a separate UI screen shall be created to ask a user what storage location, digiKam should be used for incoming multimedia data.
For example, a directory C:\FVA is selected for Local Collections as shown in the picture above. Therefore, when a user wants to integrate multimedia data after processing by FVA Tool Set, he/she shall be able to select the C:\FVA directory as a directory in UI to put the processed multimedia data in. As soon as it is done, FVA Tool Set will put multimedia data into a selected directory and the Multimedia IR system will be able to search through it.
Integration with Google Photos is implemented in the same way. First of all, it's worth noting that Google Photos shall be configured to be integrated with FVA Tool Set. Thus, appropriate file types and folders shall be set up as in the figure below:
For example, Pictures and fromDevices directories are selected for the folder settings as shown above. It means that when a user wants to integrate multimedia data after processing by FVA Tool Set, he/she shall be able to select the fromDevices or Pictures directory on the UI screen, as a directory to put processed multimedia data in. As soon as it is done, FVA Tool Set will put multimedia data into a selected directory and Google Photos will be able to search through it as soon as it synchronizes multimedia data.
| # | Name | Source | Release date | Author | Description |
|---|---|---|---|---|---|
| 1 | What is multimedia data? | repo | 01/01/2022 | Dmytro Nikulin | |
| 2 | What are Multimedia IR systems? | repo | 01/01/2022 | Dmytro Nikulin | |
| 3 | Multimedia IR systems Analyze | repo | 01/01/2022 | Dmytro Nikulin | |
| 4 | What are MultimediaOrganizers? | repo | 01/01/2022 | Dmytro Nikulin | |
| 5 | Three Cheers for Embedded Metadata | Web | 28/02/2012 | Lynda Schmitz Funrig | |
| 6 | Best Practices and Tools to Create Archival Image Metadata | Web | 28/06/2016 | Elena Toffalori | |
| 7 | Metadata | Wikipedia | |||
| 8 | Exif | Wikipedia | |||
| 9 | Google Photos | Wikipedia | |||
| 10 | Multimedia Data | Web | |||
| 11 | About digiKam | Web | |||
| 12 | What I learned as an Application Architect | GitHub | |||
| 13 | What I learned as a Delivery Manager | GitHub | |||
| 14 | What I learned as a DevOps | GitHub | |||
| 15 | What I learned as a Product Manager | GitHub | |||
| 16 | What I learned as a Software Developer | GitHub | |||
| 17 | What I Learned As a Tester | GitHub | |||
| 18 | Why I decided to create my photo organizer? | GitHub |