Serverless Backend

Use-Case Ownership Verification of User DataScope New Interface for External SystemIndustryLogistics (RIO)

Initial Situation

RIO is a leading digital brand of the TRATON GROUP and offers cloud-based solutions for the complete transport and logistic ecosystem. The RIO marketplace consolidates services for fleet managers, freight forwarders as well as vehicle rental companies to interconnect all involved parties of the supply chain management on a single platform. The services offered by RIO and its partners include digital solutions for trucks, vans, and busses independent of the specific vehicle brand. The RIO Box installed in vehicles provides a general fleet overview of a transport provider. Data such as vehicle position, its current state, and the status of the tachograph are accessible in real time. A great variety of requirements for quality assurance within the transport and logistic sector can be covered with this data: GPS tracking of the freight, accounting for the delivery of the freight at a specific time at a specific place (Geofencing), maintenance of the vehicles or monitoring the driving times to optimize route planning. This collected data can be evaluated and used by the fleet administrators and freight forwarders to improve their business strategies. To guarantee consistent data, RIO must process a continuous flow of IoT events sent by telematic devices in a fast and reliable effort. These IoT events include real time data such as wheel-based speed, fuel consumption, engine temperature, and numerous other specifications. Additionally, information about the physical structure of the vehicles may also be of interest to the platform user. This data can be released by the vehicle manufacturer and RIO must provide an interface to integrate these specifications into the platform.

Solution Approach

Technical Excellence
AWS, Serverless, CI/CD, Microservices
The LESS mindset
Mindset
Reduced costs by paying only for execution time, easily expandable infrastructure

The amount of IoT events produced by a fleet of a transport company, must first be consolidated and preprocessed before a fleet administrator or freight forwarder can extract any information. Part of this preprocessing is the secure mapping of data to the correct account of the data-owning transport company on the RIO platform so that for example data about the position of the vehicle together with data about the driver on it are only accessible for authorized personnel. A proof of ownership for vehicles is necessary to guarantee a reliable and GDPR-conform mapping of data to the corresponding account. Besides IoT events of telematic devices, data from other external systems can be provided through additional interfaces. This external master data is detached from the telematic data, but also requires a proof of ownership before a user of the RIO platform can access it.

The objective describes two different data flows that reach the RIO platform: a large continuous flow of IoT events and a smaller stream of nonrecurring master data. The larger flow of IoT events is consumed by a central component of the RIO platform that preprocesses the data and publishes it to a Kafka stream to make it accessible to a variety of specialized services. This preprocessor is constructed as a highly available component on AWS Fargate to gather data without interruption and guarantee a complete historical recording of events – even under backpressure. This component scales automatically as necessary to counteract data fluctuations and reduce operational costs. This is beneficial when considering that fewer running instances are required at night when fewer vehicles are driving and producing data while more instances should be available during peak traffic times during the day. The smaller stream of nonrecurring master data from external systems is preprocessed by a serverless backend that buffers the data in a DynamoDB and forwards it to other RIO platform services only when a proof of ownership was provided. Because this master data is usually released only once with large time intervals between data batches, AWS Lambdas were used to process data only when released which keeps costs minimal. A two-factor authentication is currently a sufficient proof of ownership for IoT events when assigning a telematic device to a digital representation of the vehicle on the RIO platform. To also gain this proof of ownership for master data provided by other external systems, a new service was implemented. This service consumes and persists the proof of ownership obtained by the association of the telematic device with the digital representation of the vehicle in order to forward this information to interested platform services. One of which is the serverless backend that can use this information to decide whether the incoming master data can be made accessible to the platform user. The Ownership-Service is a Spring Boot microservice that runs on AWS Fargate like most services on the RIO platform. This setup reduces costs, because no investment in a server infrastructure must be made. Changes of ownership are constantly monitored by the service through the consumption of a Kafka topic and are then persisted in a DynamoDB table and additionally published to an SQS-Fifo-Queue. The queue guarantees that messages are processed by another service only once and in the same order that the messages were produced.

At the moment, the serverless backend is the single consumer of the messages produced by the Ownership-Service. The AWS Lambdas will only get activated when new messages are published to the SQS-Queue or external master data is released to a connected API Gateway. This setup reduces operational costs in comparison to an AWS-Fargate-Cluster or an EC2 Instance. The external master data of a vehicle is persisted in a DynamoDB together with the ownership status. Through the use of DynamoDB Streams, another AWS Lambda is activated to forward the data to a service on the RIO platform when a proof of ownership was provided. DynamoDB Streams enable automatic retries of data forwarding, because stream events are buffered for a period of time until they are successfully processed by the AWS Lambda. This reduces the need for complex business logic while enabling a consistent data transfer to the RIO platform services even in the event of network outage. As soon as other services need to acquire the information provided by the SQS of the Ownership-Service, the infrastructure can easily be expanded by an additional publish-subscribe channel (AWS SNS) that will forward the ownership information to the components. This strategy of expanding infrastructure as needed without creating dependencies between different services is another benefit of AWS.