Access AEP as a Database
05 Jun 2022 » Platform
AEP stores a huge amount of structured data. Therefore, it makes sense that you can access that data in a way that resembles a database. In this post, I will show you a couple of ways to achieve it.
Disclaimer
Before I get into the details, I have to clarify one important concept. Adobe Experience Platform (AEP) is not a database engine. Do not forget that the main purpose of a CDP is to create advanced audiences and send them to activation channels. If you ask about the internals, sure, it uses databases to store data. However, they are not accessible and remain part of the implementation.
If you are familiar with Hadoop, a similar concept applies. While it has nothing to do with an RDBMS, you can use tools like Hive to access the data using SQL.
With that in mind, I can now proceed.
Architecture
To set the scene, let’s use the Adobe Experience Platform data flow architecture. In the next sections, I will refer back to it:
SQL
Introduction
Adobe offers a service called Query Service. It is an engine that allows you to access AEP’s datasets in the Data Lake (see architecture above) using SQL.
As I have said earlier, AEP is not structured as a relational database, which means that you cannot just write any SQL query. Besides, your XDM may be complex and not directly mappable to SQL. Before you try any wild query, check the supported SQL syntax, the Adobe-defined functions and the Spark extensions included in the tool.
That being said, having the possibility of accessing the data opens up some advanced use cases. You can use it to explore the data in search of golden nuggets (your data analysts will love it) or to connect it to a BI tool for reporting.
UI
If you have the right user permissions, you can access query services through the UI. Just head to the “Data Management” > “Queries” menu.
It is beyond the goal of this post to explain how to play with the Query Service UI. Maybe in the future.
PostgreSQL
AEP Query Service offers a PostgreSQL interface to access the service externally. Again, AEP does not run a PostgreSQL server internally, what you get is an API that is compatible with any PostgreSQL application. You can use the command-line tool, any database GUI that supports this protocol or BI tools, with your AEP data. The credentials can be found (surprise!) in the “Credentials” tab:
I have used the PSQL command that is shown in the UI and it works flawlessly.
License
Before you get too excited, I need to warn you that Query Services requires its own license. I believe there are a few options, depending on your needs. You should check whether you have one and, if you do not, whether it is the right service for you. You can always check with your Adobe customer success manager.
NoSQL
Introduction
Going back to the architecture, you will probably know that the Real-Time Customer Profile is a database with all the profiles. Through the identity graph, you can provide one identity and its namespace and get the unified profile for this identity. In a future post, I will explain what I mean by “unified profile”.
If you think about it, this behaviour is pretty similar to a NoSQL database, in particular, those that are based on key-value pairs.
UI
Well, in this case, the UI is just the “Profiles” menu entry. If you have worked a bit with AEP, you already know how to play with it:
The UI is only useful for testing purposes. You are not going to do anything useful with it in production.
API
The power of this pseudo-database lies with the Profile API. This allows other applications to access the profile data and use it as part of them. As with all AEP APIs, you need to authenticate against Adobe IO and get a bearer token.
License
As opposed to Query Services, all AEP licenses include access to these UI and API. The only limitation is the number of API calls that you can make. The standard license allows you to make 5x the number of profiles per year. You will quickly notice that this is a small number if you want to use it from the web. On the other hand, this feature can be handy in point of sales or call centres, when you want to evaluate the profile of a customer who is in front of you or on the phone.
Summary
As I have shown, while AEP is by no means a general-purpose database, in some scenarios, we can treat it as such. Some use cases can benefit from this feature. The only consideration is the AEPs license, so be sure to get the one that you need.