(Written by Kirill Voloshin & Abdullah Abusamrah)
In our previous blog posts, we have covered our server-driven UI framework called Picnic Page Platform. This framework allows anyone, including analysts and business teams, to leverage data across all of Picnic to build and ship new UI flows. This blog post explores how we’ve further evolved our framework to support more complex flows that interact with our back-end systems, persist data and more.
Before the introduction of the Page platform, all our flows were implemented by writing feature-specific code shipped with the app itself. These flows are powerful because they can be built to do exactly what is required, including communicating with the backend to write data and persist state. However, they come with a slower iteration cycle as changes have to be shipped via app updates that go through review and then are slowly adopted by the user base. This slow adoption makes backwards compatibility a critical concern, as users can stay on an older version of the app for days, weeks or even months.
With the development of the Page Platform, flows could be updated within seconds via configuration changes from the backend. The platform is great for building static experiences and, thanks to changes outlined in our previous blog post, user-interactive experiences. However, Pages lacked the ability to communicate with the backend to write data. We wanted to improve our framework even further to combine the best of both worlds: interactive UI flows with the capability to make persistent changes that can be created by everyone in Picnic and are shipped in seconds.
The Conventional Path: Using Backend APIs for Data Mutation
Usually, data mutation is performed by making POST or PUT calls to an API endpoint. If new functionality is required, a backend engineer has to create a new endpoint, implement the code necessary to perform validation on the input, retrieve any data relevant to the operation, and ensure that changes are persisted correctly in the database. One solution to allow pages powered by our platform to mutate data would be to extend Page functionality to support making HTTP requests. For example, if we want to let a customer add a recipe to their favorites from a page, it would look something like this:
That would have been a perfectly serviceable solution, but it has several drawbacks:
- Pages and our backend live in separate repositories and use different languages, making it harder to share types and data models, even though both operate on the same fundamental data. Since the models are not kept in sync automatically, it is easy to accidentally break the contract by changing a model only in one place.
- Changes to the backend logic would require the involvement of a backend developer, hurting the self-service nature of the Page Platform.
- Breaking changes to backend logic require the page using it to support both the old and the new logic, since backend deployments and page changes are not synchronized.
These drawbacks led us to look for a better solution. Taking a step back and examining the flow in the diagram above, we recognized that the processes of interacting with the Page platform and the Picnic backend share many similarities. We start out by sending a request to the service, the service interacts with the database, runs business logic and returns a response. The key differences are that the Page Platform is limited to read-only database operations and returns responses in a predetermined page format. We realized that by enhancing the Page Platform to support write operations and flexible response formats, we could use it to build APIs too.
Bridging the Gap: Enabling Backend Interactions with Tasks
This is how Tasks were born. We added a new response type to our Page Platform, called a Task, with loosened requirements on the output, allowing any JSON to be returned. To enable Tasks to write data, they needed to interact with our Java backend. As we described in a previous blog post, Page Platform JavaScript runs in the same runtime environment as Java thanks to GraalVM. We had already implemented functionality to allow Pages to call out to Java to execute SQL queries for retrieving data, by providing something called bindings. Bindings are JavaScript functions, that delegate to Java methods and wait for the results. For example, we use a binding to execute SQL queries in the Page Platform in the following way:
For Tasks, we created an additional binding called “command” that can execute various operations. Each operation has its own dedicated logic written in Java. These operations are kept as generic as possible. For example, we provide an operation to write to the Attribute Data Store, our key-value store used all over Picnic (you can learn more about it in one of our previous blog-posts). We also provide operations to interact with the customer’s cart. This adds another layer of abstraction, allowing Page writers to focus on business logic rather than thinking about the implementation details of the operations themselves. Tasks still have access to the same tools as Pages, so they can also run queries to pull in any necessary data. The flow of using a command looks like this:
So what are the advantages of this approach?
- If you know how to write a Page, you know how to write a Task. Page writers can implement entire features end-to-end on their own.
- Java developers can use their domain knowledge and focus on implementing the fundamental generic functionality and making sure it performs well.
- It lowers the cognitive load of developers implementing features, since they no longer have to worry about how things work and can concentrate on implementing the business logic instead.
- Just like our UI, updates to APIs can be shipped within seconds. Logic can be centralized in one PR, allowing reviewers to see the entire flow in one place.
- There is no need to worry about backwards compatibility because API changes and their usage can be updated at the same time.
Coming back to the original example of adding a recipe to favorites, this is what it would look like when implemented using a Task instead:
The flow may not look much different, but the replacement of the call to the backend with a call to the Picnic Page Platform is a game changer for developer independence and shipping speed.
Conclusion
Since the introduction of Tasks, we have not had to create any conventional Java endpoints to support feature development. Furthermore, we are working on migrating existing Java endpoints to Tasks. Since our framework runs inside Java, we can incrementally move functionality from Java to JavaScript. Tasks now power many of our core features at Picnic, freeing up our Java developers to focus on improving the core functionality of our backend services.
Throughout this series of blog posts, we have discussed how we developed and extended the Picnic Page Platform. As an online-only grocery retailer, we must be able to adapt and change quickly, while maintaining a high level of quality for our customers. The Page Platform has allowed us to shorten the idea iteration cycle while empowering more people, including non-developers, to contribute to the app. Tasks represent the latest puzzle piece, enabling us to build more complex and powerful user experiences quickly.
Adding Write Functionality to Pages with Self-Service APIs was originally published in Picnic Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.
