Historical Market Data Download Guide for Algorithmic Trading

·

In the world of algorithmic trading and quantitative analysis, access to reliable historical market data is essential. Whether you're backtesting a new strategy or refining an existing model, high-quality historical price data across multiple asset classes can significantly enhance your research accuracy and performance evaluation. This guide walks you through the process of downloading historical market data for A-shares, Hong Kong stocks, futures, and cryptocurrencies, using the Chanlun-PRO framework.

The system leverages established exchange APIs and integrates with database storage via the ExchangeDB class, specifically using its insert_klines method to store candlestick (k-line) data efficiently. Let’s explore how to obtain and manage this data across different markets.

A-Share Stock Market Data Download

For traders focusing on China's mainland stock markets (Shanghai and Shenzhen), two primary data sources are supported: Baostock and MyQuant (掘金).

Baostock Integration

MyQuant (GM) Integration

⚠️ Note: The latest version of MyQuant only supports up to 90 days of minute-level data download.

You can customize which securities to download by modifying the run_codes parameter, and define the earliest start time for each time cycle using f_start_datetime.

👉 Discover how to automate your trading data pipeline today.

While it’s technically possible to download full-market historical data, it’s generally not recommended due to performance overhead and storage demands. Instead, focus on downloading only the specific symbols and timeframes relevant to your strategy.

Set up a daily cron job to incrementally update your database with the latest trading data—this ensures your backtests remain current without reprocessing old records.

Hong Kong Stock Market Data

For investors analyzing Hong Kong-listed equities, the system uses Futu (富途) API for data retrieval.

Futu provides reliable access to HK market data but imposes limits on the number of queryable instruments. Therefore, users should carefully select tickers based on relevance and avoid excessive requests.

Due to rate-limiting and subscription constraints, it's advisable to:

This targeted approach helps maintain stable data ingestion without triggering service restrictions.

Futures Market Data Collection

Commodity and financial futures traders can pull historical k-line data using either TqSdk (天勤) or MyQuant.

TqSdk (Tianqin)

During the trial period, download all necessary historical futures data (e.g., rb2206, cu2205) into your local database for future backtesting.

MyQuant (GM) Futures

Both methods write data into the local database via ExchangeDB.insert_klines, ensuring compatibility with Chanlun-PRO’s analysis modules.

Ensure that your environment is configured correctly before initiating large-scale downloads. Network stability and sufficient disk space are critical for uninterrupted data collection.

Cryptocurrency Market Data

Digital asset traders can retrieve comprehensive historical data from Binance USDT-margined perpetual contracts.

This script fetches k-line data for all listed cryptocurrency pairs using Binance’s public API. It intelligently checks existing database entries via the ExchangeBinance.klines method—only fetching missing or updated records.

🔍 Important: If you've previously accessed crypto data through the web interface, the system may believe the latest data is already present, skipping historical sync.

To resolve this:

  1. Clear related database tables.
  2. Re-run the historical download script.
  3. Set up a recurring task (e.g., daily cron) to keep data current.

Supported timeframes include: 'w', 'd', '4h', '60m', '30m', '15m', '5m', '1m'.

👉 Start building your next trading strategy with real-time crypto data.

Automating this process ensures your models always train on fresh, accurate information—critical in fast-moving crypto markets.

Pre-downloaded Market Datasets via Community Sharing

To accelerate setup, pre-packaged datasets are shared via QQ groups for direct import.

VNPY-Compatible Market Data

Usage: Create a dedicated VNPY database, then import the SQL dump (existing data will be overwritten).

Partial Cryptocurrency Dataset

Partial Futures Dataset

These snapshots allow rapid deployment for testing and development without waiting for lengthy downloads.

Frequently Asked Questions (FAQ)

Q: Can I use Baostock for tick-level or second-by-second data?
A: No. Baostock only provides data at 5-minute intervals and does not offer tick-level granularity or technical indicators.

Q: Why isn’t my cryptocurrency history updating even after running the script?
A: This usually happens if recent k-line data was fetched via the web UI first. Clear the relevant database tables before re-running the historical sync script.

Q: Is there a way to extend beyond MyQuant’s 90-day minute data limit?
A: Unfortunately, no. You’ll need to combine multiple sources or rely on lower-frequency data (e.g., daily bars) for longer historical spans.

Q: How often should I run the data sync scripts?
A: Daily execution via cron jobs is recommended to maintain up-to-date databases for accurate backtesting.

Q: What’s the advantage of using adjusted close prices for A-shares?
A: Adjusted prices account for dividends and splits, ensuring consistent price continuity over time—critical for long-term strategy evaluation.

Q: Can I contribute my own historical data to the community package?
A: While not covered here, community-driven sharing via QQ groups allows users to exchange verified datasets—always verify integrity before importing.

👉 Unlock powerful trading insights with advanced market analytics tools.

By following best practices in data selection, automation, and maintenance, you can build a robust foundation for quantitative research and algorithmic trading success. Whether working with traditional equities or fast-paced digital assets, having clean, structured historical data is the first step toward developing winning strategies.