Does your business need to gain better data insights? Would you like to collect, organize, and activate data from any source, be it online, offline, mobile, and more? Then you need a data management platform, or DMP.
Let’s start with a brief introduction to DMPs. Data management platforms allow you to organize, collect, and activate audience data from any source. Through this, a DMP will add value to your business by providing insights about your customers.
Today, you can buy a DMP from a number of vendors. However, the cost usually ranges from $80K to over $1M for large implementations.
But don’t fret—you have another option. You can build one yourself.
In this post, I’m going to explain how a data management platform works, features of a DMP, and the architecture for building a DMP.
How Does a DMP Work?
A DMP’s job is to collect raw customer data from a number of sources such as desktops, mobile websites, mobile applications, analytics tools, point of sale devices, and social media. DMPs collect raw data by using the following:
- Tags placed on websites and in mobile apps (usually in an app’s software development kit [SDK])
- Application program interfaces (APIs)
- Server-to-server integrations
- Cookie syncing and data enrichment from third-party data sources
- First-party data onboarding
DMPs make use of data that falls under three categories: first-party data, second-party data, and third-party data.
First-party data is acquired when a company collects data directly from their customers, using specific behaviors such as clicks, downloads, likes, and customers’ demographic information. Second-party data is acquired when a company buys data from other organizations. Second-party data is basically first-party data that’s been collected directly by someone else.
An aggregator provides third-party data and collects and packages it from a variety of sources.
Both second- and third-party data have the same advantages as first-party data, but they can provide further insights. The DMP then organizes and structures this data to form information that can be applied to a specific class of customers—for example, mothers between the ages of 25 and 35. This is one of the many applications of a DMP.
What Features Your DMP Should Provide
This largely depends on the needs of the stakeholders, but here I’ll list some features that a basic DMP should provide.
Long-Term Support
Using a DMP requires long-term support beyond the implementation phase. If you choose to build your own DMP, I advise that you create a team of data scientists who have skills in areas such as machine learning and artificial intelligence and experience with statistical models.
Data Collection and Organization
The data management platform needs to aggregate and organize first-party, second-party, and third-party data from any source. You should build your DMP so that it can ingest data in real time and in batches. This will require developers to code tags for web properties, SDKs for mobile apps, and batch imports for offline data.
Second- and Third-Party Data
The DMP should allow access to second-party and third-party data inside the platform. The DMP should preferably facilitate this relationship, rather than force the users to find interested companies. To access this data, you’ll need to build long-term relationships.
Audience Building, Insights, and Reporting
The DMP needs to allow the user to build complex audiences using demographics, content consumption, and the audience’s interests and actions. Furthermore, a good DMP should allow the user to contrast audiences against first- and third-party data to learn greater insights.
Back-end developers need to create data models so that they can provide these analytics while front-end developers design and create user interfaces (UIs) and data visualizations.
Data Security
This is a critically important feature. Employ someone who understands data privacy and keeps up to date with all laws and regulations. It’s important that you use and transfer data in a way that is in line with recommendations from regulatory bodies.
Architecture
Now, let’s look at the architecture of a DMP.
The Data Collection Layer
Client-Side
The client-side should focus on collecting data and communicating it to the server side of the data collection layer. SDKs, JavaScript trackers, and JavaScript frameworks such as AngularJS or libraries like ReactJS collect and communicate the data.
Server Side
The server side’s role is to provide endpoints that are responsible for receiving the data from the client-side libraries. It’s simply used for logging data and queuing the data up for the next layers to process.
The Data Normalization and Enrichment Layer
The data is then received from the data collection layer endpoints, and thereafter the data is normalized and enriched. Thus, what follows is called the data normalization and enrichment layer. Here, redundant data is deleted, the DMP’s data scheme is implemented, and if necessary, data is enriched with additional data points.
The Data Storage, Merging, and Profile Building Layer
Next, the new data needs to be merged with existing data, stored, and used to create user profiles. Profile building is important as it’s used to turn the collected data into events and profiles, which are necessary for audience segmentation.
User profiles may have a number of identifiers. For example, cookies or device identifiers and persistent identifiers that are pseudo-anonymized, like hashed out usernames or email addresses.
A profile consists of user attributes (e.g., home location, age group, gender, etc.) as well as events (e.g., page views, forms filled in, transaction, etc.). Collections and databases store the events.
The Data Analysis and Segmentation Layer
The main purpose of a DMP is to analyze data and create segments for specific audiences or customers. A segment allows marketers to break down data. This makes it easier for them to allocate the data to specific groups of potential customers.
Besides the layers, a DMP should allow for segments to be used in campaigns using audience export. Furthermore, DMPs need a UI so that users can interact with it.
The data management platform uses APIs, and they fall into the following categories:
- Platform API: needed to create, remove, update, and delete objects
- Reporting API: provides information about the data using reports
- Audience API: allows client libraries to query in real time whether a given visitor belongs to the audience or not
- Data ingestion API: used for importing the segments or other data from third-party platforms
Conclusion
In summary, this guide explains how data management platforms work, which features are necessary in DMPs, and their architecture. Now that you know about DMPs, you should feel more confident in assessing whether building a DMP fits your needs!
Explore the DataOpsZone blog to learn more about DMPs, get solutions on issues you’re facing in data management, and learn how to use some tools/technologies for solving these issues. Then you can easily decide if it’s the right thing for you.
This post was written by Daniel de Oliveira. Daniel is a front-end designer, writer, and data analyst. He has experience working with Flutter, Ionic, and Django.