How ForInvest Deliver ~3TB Historical Market Data in Milliseconds

Nowadays, the popularity of financial data has been increasing dramatically. Almost everyone is following stocks, currencies, and cryptocurrencies on a daily basis. Customers wish to access financial market data in a fast, reliable, and uninterrupted way to follow the pricing of stocks, make analysis within graphs, and more. For this reason, financial data vendors take a crucial role to transmit not only real time but also historical data to customers in a rapid and reliable way. 

As a leading fintech company, one of our critical missions is to deliver historical market data to your customers no matter what! To accomplish this; 

  • ForInvest’s DevOps Engineers decided to set up a distributed, replicated, secure and self-managed mongoDB powered by AWS.
  • ForInvest Software Engineering team developed high-quality, fast, and secure REST APIs.

ForInvest is a global technology company that develops omnichannel fintech solutions. ForInvest specializes in broadcasting financial data with a full range of products using mobile devices and fixed stations providing services to all parties involved in the financial markets. Its client base consists of banks, brokerage houses, foreign exchange bureaus, and other financial institutions, Internet service providers and portals, corporations, and private investors.

ForInvest, have been delivering historical market data since the late 1990’s. Our Engineering team implemented highly available, modern, easily adapted REST APIs to deliver data to customers around the world. 

Some of the historical data types we provide to our customers are:

  • Stocks
  • Indices
  • Bonds
  • Currencies
  • Cryptocurrencies
  • Commodities (Metals, Energy)
  • *BIST Equity Market
  • *BIST Derivative Market (VIOP, Options, Futures)
  • *BIST Indices

*Borsa Istanbul briefly referred as BIST

I’ll mention architectural design and infrastructure set up to clarify how ForInvest delivers a serious amount of historical data to customers in milliseconds.

Architectural Design & Infrastructure Setup

When you deliver historical market data to your customers at scale, you need to consider a few things: 

  • High Availability
  • Scalability
  • Replication
  • Rapid Delivery
  • Secure Communication

The following diagram includes every aspect mentioned above.

High Availability

To provide HA, ForInvest DevOps team has deployed containerized mongoDB instances distributed along many availability zones in AWS.

Scalability

To maintain consistent service availability under high load, ForInvest DevOps team has implemented automated scaling in AWS

Replication

To maintain the same data set along with distributed mongoDB instances, our engineering team configured mongoDB in replica set mode. Thanks to the replica set, data has always been synchronized for each of the mongoDB instances.

Rapid Delivery

MongoDB indexes structure is applied to our REST APIs to efficiently execute queries. Therefore, MongoDB uses a related index to limit the number of documents it must inspect.

Secure Communication

Last but not least – security! Our DevOps team configured a replica set with keyfile authentication. Enabling keyfile auth requires not only user authentication but also key-based internal authentication of replica set members.

Conclusion

It is very important to reach historical data in an uninterrupted, fast and reliable way from the customer’s perspective. Historical data enables the tracking of improvement over time which gives key insights. These insights are essential for driving a business. One of our primary focus is to transmit historical data to our customers by leveraging latest technology with our financial services.

ForInvest continues to evolve by combining its 30 years of industrial know-how with new generation innovations. Discover our innovative, compact solutions that lead the finance world to be a pioneer in the industry.