Event detail records (EDRs) are a persistent record of subscriber activity generated every time a subscriber uses a telco service. They can be processed by analytics systems to produce business intelligence reports, including service usage analysis, average revenue per user (ARPU), churn risk, identifying potential areas of revenue loss, and even new opportunities to increase revenue. These reports provide important insights to ensure service design is optimized in line with business goals.
Whether implementing new, modern 5G services or monetizing cloud, IOT, and partner services, telco IT teams are discovering it is almost impossible to build metrics that quantify their business goals without being able to rely on an efficient, secure, and scalable BSS enhancement platform. A good BSS enhancement platform needs to provide the capability of building data insights easily – without the baggage of investing in excessive infrastructure, or long months of effort integrating with legacy data warehouse systems.
Many telco IT teams are being challenged to build modern data analytics systems by other teams inside their telco organization, such as commercial, sales, marketing, data teams). These other telco teams outside of IT need to accurately measure subscriber success using newer 5G services.
Unfortunately, IT teams trying to use EDRs usually find it difficult to wrangle sufficient performance out of their existing on-prem charging system. Legacy chargers don’t allow these teams to deliver modern data analytics very easily. Procuring the infrastructure, along with spending significant time and money to integrate with legacy data warehouse systems, makes getting at modern, data-informed insights just too slow and too expensive to reasonably deliver.
In this blog post, we describe how Totogi enables a five-step process that IT teams can follow to build a modern data analytics dashboard (example shown below) for 5G services.
Follow our step-by-step guide for building this example dashboard using EDRs generated by Totogi Charging-as-a-Service. This guide will also show how data analytics can be enhanced through the simple integration of other AWS services.
Totogi Charging-as-a-Service EDRs
Totogi Charging-as-a-Service generates two types of EDRs for subscriber activities: charging events and subscriber management.
1) Charging events are related to charging requests and modifications to the monetary balance, including purchasing a plan or a payment top-up.
2) Subscriber management events are related to subscriber account management, including account activation, deactivation, and device activation.
Events generated by Totogi Charging-as-a-Service are stored in the optimized row columnar (ORC) format in AWS S3. This format provides a highly efficient way to store EDRs with improved performance for processing data, as compared to traditional formats, such as CSV, JSON, or XML.
In ORC format, data is stored in columns that are compressed, leading to a smaller disk read and reduced file transfer. ORC format also has built-in index and aggregation values (such as min/max) to improve read performance by reading minimal rows.
AWS also provides automatic, asynchronous replication of objects across AWS S3 buckets. This is helpful because telco IT teams can utilize these events and build rich data-informed insights using either AWS products or in-house business intelligence systems.
Now, let’s learn about the five key steps that telco IT teams can follow to build a data-informed insights dashboard using EDRs in combination with AWS services.
Step 1: Get started with Totogi Charging-as-a-Service
First, sign up for Totogi Charging-as-a-Service in the AWS Marketplace using an AWS account. Here is a 90-second video to help you sign up:
After signing up successfully, the account holder will receive credentials to sign in to Totogi Charging-as-a-Service.
Sign in, then create and launch a plan with required services, allowances, and price configuration. Here is a 2-minute video to get started with the plan designer:
From here, the Totogi account holder can use this guide to create accounts and devices, credit an account, subscribe to a plan, or simulate usage.
Step 2: Replicate EDRs to a new AWS S3 bucket
Create an S3 bucket in the same AWS account, then provide permission using the bucket policy to allow Totogi to replicate the EDRs. The Bucket Name and Provider ID are shared with Totogi so that automatic replication can be enabled.
Next, EDRs are replicated to customer-owned S3 buckets in 15-minute intervals, with increased frequency in case of high transaction volume.
Shown below is the directory structure of the S3 bucket. EDRs are further grouped into account, billing, charging, notification, plan, rating group hierarchy, and tenant to easily identify and process. Inside each EDR group, subfolders will be created with year, month, and day structure in the directory.
| – type=CHRG/
| – type=PLAN/
| – year=2022
| – month=08
| – day=27
Step 3: Create a data catalog
The next step is processing, transforming, then organizing the EDR data.
AWS Glue is a serverless data integration service that makes discovering, preparing, moving, and integrating data from multiple sources more streamlined for analytics, machine learning (ML), and application development.
Follow along with the GIF below to create a Crawler in AWS Glue to crawl and populate an AWS Glue data catalog.
Follow along with the instructional video below to create a Crawler in AWS Glue to crawl and populate an AWS Glue data catalog:
During the first execution, the Crawler will process a subset of the data and infer the schema. It compares schemas from all subfolders and files and then creates one or more tables.
Shown below is the snapshot of the schema created by a Crawler:
Step 4: Query event detail records
Next up, AWS Athena can provide an interactive query service to analyze data from AWS S3 using standard Structured Query Language (SQL). These queries are executed on the tables that AWS Glue created in Step 3.
Here is the sample query for viewing the top 3 data consumers:
Telco IT teams can also create views with the tables, along with necessary additional columns or fields, for analytical purposes.
Step 5: Create a data insights dashboard
Now we’re ready to share data-informed insights with other teams inside your organization. For this step, we use another AWS service: AWS QuickSight. QuickSight supports creating serverless dashboards by connecting data in AWS S3, and querying using AWS Athena. From here, telco IT teams can build feature-rich dashboards to share with management and other teams.
Telco IT teams can create the data set by selecting the data source as Athena, then selecting the catalog, database, and table/view names already generated in the previous steps. AWS QuickSight automatically selects the data type during data set creation and provides the capability to modify the data type as required.
Follow along with the instructional video below to modify data type and create the visualization.
AWS QuickSight will display all the fields from the table/view in a dashboard, so that now other internal teams or external subscribers can select the specific fields (for example, date and session id). Then, QuickSight will automatically create the best graph for the data analytics dashboard that offers data-informed insights to any telco team. From this point, IT teams can alter the fields and visualizations within the data analytics dashboard as required, with zero dependencies on IT.
BONUS STEP: Integrate data insights using GraphQL APIs
Totogi provides access to EDRs using two GraphQL APIs (getEventDataRecordsByDevice and getEventDataRecordsByAccount). By using these two APIs, IT teams can retrieve EDRs to build usage dashboards, then integrate the dashboards with their own in-house CRM or Self Care application.
The responses from GraphQL API queries of these two APIs can be optimized using filters, such as by device, start time, end time, or by type of event.
Get more detailed instructions on using GraphQL APIs for EDRs in our How-to guide on EDRs.
Explore data insights on your new dashboard
Congratulations, you should now have your very own data-informed insights dashboard, created with EDRs generated by Totogi Charging-as-a-Service, along with a handful of AWS services in just a few hours.
Want more developer content
Could have a nice CTA
Link to another resource or blog when we have one