The Sharding pattern is when a datastore is separated from a single storage instance into multiple instances called shards. Data is then divided according to some form of sharing logic. Queries are executed against the shards. It’s up to the database technology implementing the shards to perform the queries in an optimized manner.
One type of sharding logic is when database administrators (DBAs) segment data according to values in a particular field in the database. An example of such separation is dividing data among the shards according to the first letters of the last name.
Another type of sharding separates data according to a segmentation logic that ensures that disk capacity is consumed equally among all instances.
Pros
- The sharding pattern is well suited for large, distributed enterprise applications.
- Sharding allows for the fast execution of a command or a query.
- Storage segmentation, which is a key feature of the sharding pattern, enables the physical infrastructure to scale in a controlled manner.
Cons
- The sharding pattern requires that DBAs have both specific domain expertise and experience with the best practices of the database technologies in play in order to manage the sharding segmentation effectively.
- Shards distributed over a large number of geolocations can be susceptible to performance degradation due to excessive network traffic.
- Some database technologies are better suited to the sharding pattern than others. Thus, you need to choose wisely.
- Added hardware means a higher total cost of ownership of the service.
Conclusion
Separating data using the Sharding pattern is well suited to large distributed applications. Large enterprise applications depend on fast data access. Logically segmenting data according to a data table’s key or a database’s storage capabilities executes queries with a fine grain of precision. Searching for data according to shard is faster. Also, scaling becomes more efficient.