As an online grocery retailer, we operate in a complex environment that requires us to adapt on an ongoing basis to changes in customer behavior, our operations, legislation etc. In order to do so adequately, we need to be able to ship changes to our apps often and with little lead time. For example, when the government introduced lockdowns during COVID, we were still allowed to deliver groceries but not alcohol after 20:00, a restriction that came into effect practically overnight.

Over the years, we have changed our way of working several times, with the aim of optimizing our ability to ship changes to our app quickly.

On the mobile side, in 2019 we started migrating our native Android and iOS apps to React Native to reduce the amount of platform-specific code we need to write for each feature. As part of this change, we adopted a more modular app architecture (inspired by Uber’s Riblets) in order to reduce the amount of sweeping changes. Halfway through 2022, most of the core functionality in our app was powered by code written in React Native. We had cut the lead time for most features almost in half by reducing the amount of code to write and unifying our architecture. As a side-effect, these improvements decreased opportunities for bugs to manifest themselves, allowing us to allocate more time to feature development. In addition, the adoption of React Native also brought our Android and iOS engineers closer together, transitioning from separate silos into a unified team that identified themselves as Mobile engineers working across the client-side stack.

In the backend, we developed a real-time rule evaluation service that enables anyone in Picnic with some basic coding skills to create and modify rules that integrate with our systems landscape. Rule evaluations are triggered by events occurring in our systems (e.g. an order being placed by a customer, an article being picked in a warehouse, a vehicle leaving a hub) and trigger actions in others (e.g. sending a push notification, changing an in-app configuration). This decouples the ability to iterate and experiment with the configuration of existing features from having engineering capacity available. As of today, we have hundreds of such rules running across markets, powering not only the consumer experience but also core operational processes.

Introducing the Picnic Page Platform: A New Era of Flexibility

Being Picnic, we didn’t stop there. With around 30 engineers working on our grocery shopping systems landscape on a day-to-day basis, we’re a relatively small team with big ideas and ambitions. Even with a cross-platform codebase and the rule engine in place, incremental improvements to our apps still required us to write and ship (cross-platform) code. For new features, not only did we have to implement new screens in the apps, we also had to implement a significant amount of (boilerplate) backend logic to persist and aggregate data to be presented on these new screens and serve them to the clients. All in all, the lead time for shipping even a seemingly trivial change, could still be a couple of weeks.

Working in Software Engineering, we all know that shipping is only the beginning. Accounting for the time it takes for people to install a new version of our app, it could take months for a feature to reach the majority of our customer base, leaving us with the burden of maintaining backwards compatibility. And given that the features that we ship often need to be configurable to empower operators and that it is hard to anticipate the required configuration space upfront, reaching a “final” delivery of a feature can take several iterations, stretching the lead time for adoption by our entire customer base even further. Being an ambitious company that wants to move fast, it was clear that we wanted to do even better.

DETP, a framework for feature development

Analyzing the work that we do on a daily basis, we realized that for many features that we build and deliver, we solve a variant of what I will coin the Define, Extract, Transform & Present (DETP) problem:

• Define: the experience that we want to present to customers is represented by data that we have or that we ask our customers to provide. This data ranges from entities that are core to the Picnic proposition such as deliveries, delivery slots and products, to more ephemeral data such as whether a customer has used a certain feature recently or not.
• Extract: in order to present the data, we need to extract it from the various data sources.
• Transform & present: once extracted, the data needs to be transformed into something meaningful to be presented to the user. Note that presentation is not necessarily purely visual, and depends on the platform that the user is on. As an example, some of our customers depend on assistive features such as screen readers or devices that lack a visual interface such as voice interfaces.

In solving this problem, we do a lot of work that is essentially a variation of work we have already done earlier. On the server-side, we define a data model and implement infrastructure that allows us to read and write that data. On the client-side, we implement logic to query the API exposed by the server — sometimes combining data from various endpoints — and then transform that data into a visual representation to be shown to users of our app.

Having realized these similarities between the features that we were building, we started to think about a better, more generic solution to the problem outlined above that would allow us to iterate faster. To begin with, we set a Big, Hairy, Audacious Goal: to enable any person working at Picnic to deliver iterative app changes to our customers without requiring time on our engineering roadmap.

Building a Fully Integrated Application Development Platform

In order to get there, we set out to work on an application development platform that solved our needs. Based on our experiences with smaller-scoped server-driven UI features and over many iterations, we started to settle on a vision for a platform consisting of the following components:

• A central persistence layer to define and persist attributes over arbitrary dimensions
• A generic API to extract and combine data from various data sources
• A programming environment that allows for transformation of this data into a definition of a view to be shown to the customer
• A rendering engine that presents this view in accordance with the requirements of the platform that the end-user is using
• A set of tools that supports Picnic team members to configure the experience of our end-users

In the next few weeks, we will detail each of the components outlined above in follow-up articles written by the team. In these articles, we will explain the challenges we have faced and the choices we have made while building them.

The Results: Faster Shipping, Real-Time Updates, and a Unified Team

After three full years of investing in this platform, we couldn’t be happier with our approach and the advantages that we gained by adopting this approach:

• By now, over 75% of the customer experience is running on our platform. All these screens benefit from any improvement we add to the platform. This has allowed us to increase consistency, significantly improve performance, refine our analytics data collection, and step up our approach to monitoring and observability to name but a few.
• Our investment in developer tooling and experience has brought business stakeholders and engineers closer together. Rather than communicating via lengthy requirements documents, business analysts and engineers now get together to co-create the data models and queries that power our features.
• Since changes are immediately deployed and available to end-users without requiring an app update, we are able to ship non-trivial changes to our app more than 5 times per day on average. These changes can go from commit to customer in under 15 minutes!
• Thanks to extensive documentation and developer tooling, any engineer at Picnic is able to ship a change to the app. Before, we were limited in our ability to deliver end-to-end features by the number of mobile engineers on our teams. Now, teams consisting of only backend engineers are shipping material changes to the app. On top of that, we see that many of our business analysts have enough basic programming skills to implement features end-to-end.
• Once a feature is delivered, incremental change and experimentation can be achieved without engineers involved. Business analysts have added features such as alternative product suggestions and related product suggestions to our Product Detail Page completely independently.

With our platform taking away the busywork in feature development, our team members can focus on the interesting parts of our daily challenges: delivering value and serving our customers’ needs by making the best groceries shopping experience, one commit at a time. I couldn’t be more excited by the future we have ahead of us!

This article is part of a series of blogposts on the architecture behind our grocery shopping app. Hungry for more? Chew off another bite with any of our other articles:

• Define, Extract and Transform inside Picnic’s Page Platform gives a high-level overview of our journey towards a platform that enables querying data and transforming it into rendering instructions for the client.
• In Enabling rapid business use case iteration with Apache Calcite, we go into great detail of why we chose Apache Calcite as the tech behind our query engine, and how we are using the technology.
• Picnic’s Page Platform from a Mobile perspective: enabling fast updates through server-driven UI details our approach to implementing server-driven UI in our mobile apps.
• In Tackling Configuration: creating Lego-Like Flexibility for non developers, we discuss the configuration tools we created on top of this architecture, allowing non-engineers to change the configuration of our Store.

Stay tuned in the coming weeks while we publish more blogposts in this series.

Faster Features, Happier Customers: Introducing The Platform That Transformed Our Grocery App was originally published in Picnic Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.