Big data is all the rage, and as companies scramble to gather as much data as they possibly can, the eventual outcome is typically that you end up with more data than you know what to do with. So in this lake filled with files and objects, it's very easy for data to get lost. This is where metadata comes into play. In a simple way, metadata is additional information about your data, both automatically and manually created. For example, in the case of a picture, the automatic metadata can include the exact time the picture was taken, the type of camera that was used, the GPS location and the resolution, whereas some of the manual metadata can include the name of the author and a description. Having this type of metadata attached to every piece of data you store is crucial, especially as you scale up. One challenge that often presents itself is how to store this metadata, since most file formats don't allow you to add this metadata to the file itself. Using a database One common
Posts about technology, DevOps, the cloud and more.