Driving Ecommerce excellence through software development
Learn how we helped a scaling eCommerce platform sync thousands of different products and prices from hundreds of catering suppliers with thousands of restaurant owners.
Talk to Jeroen, our Accounts director, to see if our ready-to-code dev teams are a fit for you.
When it comes to software development, sometimes a challenge can’t be solved by an out-of-the-box solution. Instead, it requires thinking outside the box.
That’s exactly what was required when Proshore helped a scaling eCommerce platform sync thousands of different products and prices from hundreds of catering suppliers with thousands of restaurant owners.
Handling large volumes of product data
Orderxl, formerly Blinqed, is a B2B eCommerce application that connects catering suppliers with restaurant owners and streamlines the ordering process. It gives buyers the best possible prices, and sellers the flexibility to make their pricing more dynamic.
Proshore initially came on board to provide Orderxl with the technical expertise they needed to make their application a reality. One of the initial challenges was to import huge volumes of historical product data and pricing from a legacy system into the new software. As their business rapidly scaled, our attention shifted to how to handle increasing volumes of incoming data and accelerate pricing updates for users on the front end.
Buyers want the best possible deal on their catering supplies, so the application provides dynamic pricing, which is controlled by the wholesaler. However, due to the sheer quantity of data, the time taken to parse this information was creating a bottleneck between the sellers’ enterprise resource planning (ERP) systems, and the price changes being reflected on the front end of the website.
That’s when Proshore’s development team as a service stepped up to provide an innovative way of speeding up pricing changes on the front end.
Capacity to handle over 750,000 pricing changes per day
Every hour of every day, product information, including pricing, has to be pulled from multiple ERP systems. This is then reflected on the front end of the website. Data is dynamic and is completely based on buyer and seller activity.
However, when a wholesaler has over twenty customers and each one of those has a bespoke catalog of thousands of products, the amount of data flowing into the system soon stacks up. In terms of pricing change records, the system can handle over 750,000 record updates per day. And it was the parsing of all that product data for each customer that was slowing down the system.
Importing and reading huge volumes of data meant that customers didn’t always see the most recent prices in real-time, there would be a delay in loading times – sometimes nearly 30 seconds per order page and occasionally crashing when buyers searched the product listing.
To tackle the problem:
- We requested that wholesalers only provide the updated product data, rather than submitting the entire product catalog.
- With the help of a developer based in the Netherlands, we were able to filter the data so that all the relevant changes from an ERP system could be sent in a single file.
- We upgraded the AWS relational database based on need. Performance was monitored and queries were optimized to provide better scalability.
- We also split the database into Reader and Writer instances, making it quicker to read data needed for the front end.
These improvements reduced the amount of data flowing into the system which needed to be parsed. In turn, this increased the speed at which price changes appeared on the front end.
In fact, over time, Proshore’s development team reduced the time taken to parse one dataset from 4 to 5 hours down to under 10 minutes!
We can help you get started in 15 minutes.