Design an E-commerce website like flipkart.com

In this entry we will discuss a design for E-commerce website like Flipkart. Before going into the design details, following are the list of Features.

List of Features:

1. User can sign-up, sign-in,search a product,view categories, view products,add a product to the cart, check out  products, add product to wishlist, share wishlist and what not...Here we will divide the list of features into following high level user stories:

• Customer/User related stories eg cart, wishlist etc.
• Admin related stories eg: ON/OFF a new product, canceling an order, adding offers etc. This all features will be wrapped into a Admin portal.
• Product search and categorization related stories.
• The system should support 1 Million requests per second.
• The system should be high scaling and availability.

In this design our focus would be on Customer/User Related tasks and Product search and categorization related tasks. Our system should support 10,000 customer/second and should be able to serve 100 Million products over different categories.

For product search, we can use Solr- an open source enterprise search platform with cache.Whenever a product request comes, load balancer will search a product in Solr, if the product exist detail of the product returned without querying the database. We are going to cache 40% of our product in Solr. Assume each of our products is of 10MB including image,description and other metadata.100Million product would be approx 1000TB or more, So we need to cache 400TB data in Solr. Considering a machine can store 10TB data in hard disk,  we will need 140 machine for data and 100 machine to add redundancy.

Design Strategy:

To build high scaling application with ~zero downtime ~high availability one of our strategy would be to use Microservice based approach. Microservices is an approach to application development where a large application is built as a suite of granular services. Each service supports a specific business goal and uses a simple, well-defined interface to communicate with other services. High level design is as follow:

In above design, we are storing the session state at the client and and session data in in-memory database using session-service. The shopping cart for each user is stored as a restful resource on the server.

If you have your data sharded across multiple databases. lot of HTTP server, Cache server responding the request, you need to have a provisioning server to auto scale the capacity dynamically by setting up servers as need or removing in case of reduced load. In this case its better to go full stateless and leave the shopping cart with client session. Then once user hits check-in, shopping cart should be persisted on database. The key about the rest is that every request the client makes has to contains all data to identify itself. I would suggest reading about  OAuth.