Open Datasets & Machine-Readable Feeds
How do point-in-time AI infrastructure and physical capex datasets ground-truth corporate spending?
Direct Answer: Alongside federal disclosure feeds, Gemral Edge maintains a point-in-time physical infrastructure dataset holding 430,043 public records across 7 datasets (as of 2026-10-25): data center and industrial footprints (225,526 records, 2008–2026), environmental permits (454 records, 1984–2025), electric grid and substations (101,852 records, 2007–2026), regional electricity demand (92,837 records, 2019–2026), equipment import shipments (740 records, 2013–2026), company capex and filings (2,812 records, 2009–2026), and satellite site indices (5,822 records, 2018–2026).
What datasets are available in the public index and download feeds?
Canonical open data downloads for financial researchers, econometricians, and algorithmic agents:
| Dataset Title | Data Grain | Coverage Window | Direct Download |
|---|---|---|---|
| US Congressional Stock Trading Disclosures | 1 row per transaction | 5 Years (2021–Present) | Download CSV |
| Congress Stock Trade Disclosure Latency | 1 row per disclosure + days-late | Rolling (most recent 5,000) | Download CSV |
| Federal Reserve & Treasury Net Liquidity Metrics | 1 row per settlement day | 2 Years (2024–Present) | Download CSV |
How do public federal intelligence datasets integrate into open data architectures?
Gemral Edge provides open, standardized, machine-readable data feeds synthesized from primary United States federal government records and statutory disclosures. Designed for quantitative researchers, financial data scientists, investigative journalists, and autonomous AI agents, our public datasets eliminate the operational overhead of scraping and cleaning complex government portals while maintaining strict primary-source provenance.
What are the four foundational federal datasets underlying the platform?
Our public repository currently publishes four core datasets updated on continuous rolling schedules: (1) Congressional Stock Trading Disclosures, capturing all equity, option, and asset transactions filed by sitting members of the U.S. House of Representatives and U.S. Senate under the Stop Trading on Congressional Knowledge Act of 2012 (STOCK Act); (2) Federal Defense and Technology Prime Contracts, tracking newly obligated procurement awards across the Department of Defense, DARPA, and federal agencies resolved to listed parent corporations; (3) Senate lobbying disclosure Corporate Lobbying Disclosures, indexing corporate lobbying spend, registered lobbying firms, client entities, and targeted legislative issue codes; and (4) Macroeconomic Liquidity Metrics, calculating daily net liquidity flows derived from Federal Reserve total assets, the Treasury General Account (TGA), and the Overnight Reverse Repurchase Agreement facility (ON RRP).
How does the dual-licensing framework balance CC0 public domain and CC-BY attribution?
To maximize research freedom while preserving documentation integrity, Gemral Edge implements a transparent dual-licensing framework. All raw statutory records, transaction data points, and government filings originate from official public domain sources and are released under the Creative Commons CC0 1.0 Universal Public Domain Dedication. Users may copy, modify, distribute, and execute commercial or non-commercial analyses without requesting prior permission. All accompanying schemas, metadata descriptors, and analytical methodologies authored by Gemral Edge are licensed under Creative Commons Attribution 4.0 International (CC-BY-4.0). We request simple citation attribution: Data: Gemral Edge (https://www.gemral.com/edge/datasets).
How are programmatic CSV exports and agentic ingestion configured?
Every dataset is accessible via direct, permanent HTTP endpoints delivering RFC 4180 compliant UTF-8 CSV streams with full Cross-Origin Resource Sharing (CORS) headers. AI models, automated pipelines, and statistical environments such as Python Pandas or R can stream datasets directly without authentication barriers or rate-limiting friction.
What standardized column normalization standards ensure quantitative modeling compatibility?
Every dataset distributed across the Gemral Edge open repository adheres to strict schema normalization standards. All transaction records, procurement obligations, and lobbying filings feature ISO 8601 UTC timestamps, standardized ISO-3166 jurisdiction codes, verified Legal Entity Identifiers (LEI), and Commercial and Government Entity (CAGE) codes. Numerical values—including obligation amounts and transaction valuation brackets—are normalized into standard floating-point figures, allowing quantitative researchers to ingest clean tabular feeds directly into Pandas, NumPy, and Polars without custom data wrangling scripts.
How are cryptographic validation hashes and immutable changelogs maintained?
To maintain absolute scientific reproducibility, every released dataset is accompanied by a SHA-256 cryptographic checksum and a timestamped release manifest. When official government revisions occur, our data pipeline issues versioned snapshot archives, ensuring that backtesting researchers can accurately reproduce historical quantitative experiments without risk of look-ahead contamination or silent schema drift.
How do direct Parquet and DuckDB connectors optimize analytical query throughput?
For quantitative researchers analyzing millions of historical procurement rows and multi-decade legislative filings, Gemral Edge provides columnar Apache Parquet files optimized for high-speed analytical engines like DuckDB and ClickHouse. Columnar partitioning by fiscal year and industry sector reduces query I/O by up to 90%, enabling instantaneous aggregations and statistical regression analysis on commodity hardware.
What licensing terms and academic research exemptions govern open dataset redistribution?
All public datasets published across the Gemral Edge open repository are distributed under permissive open-data licensing frameworks that permit unrestricted commercial and academic utilization. University research departments, economic think tanks, and independent investigative journalists can freely download, redistribute, and integrate our historical datasets into published peer-reviewed scholarship with standard citation attribution.
How do automated daily validation scripts guarantee zero data loss in historical repositories?
Our data engineering infrastructure executes continuous checksum reconciliation scripts that compare historical database partitions against immutable raw government filing mirrors. By verifying record counts, sequence continuity, and field boundary constraints across multi-decade archives, the platform guarantees that historical datasets remain pristine and free from corrupt or missing records.
How do RESTful pagination and cursor-based streaming protocols optimize bulk downloads?
Our public dataset distribution architecture implements RFC 5988 cursor pagination and streaming ndjson protocols, allowing quantitative engineering teams to ingest millions of historical rows with deterministic state resumption and zero memory overflow risks on resource-constrained compute instances.
How do data schema versioning policies ensure backwards compatibility for automated quantitative pipelines?
All Gemral Edge public data endpoints adhere strictly to semantic data versioning protocols. Schema changes, field deprecations, and type expansions are published with multi-month migration runways and backwards-compatible JSON schema definitions, guaranteeing that institutional trading algorithms and production ETL pipelines operate with unbroken continuous ingestion.
Automated regression test suites continuously benchmark query latency across all historical partitions to maintain deterministic sub-second delivery SLAs.
When institutional research teams evaluate alternative data providers for AI data center power demand and capital expenditure verification, relying solely on quarterly corporate financial statements leaves a multi-month visibility gap. Physical construction leaves an observable trail across state and county environmental permit and land-use records, federal energy and trade statistics, company filings with the securities regulator, and open map data and public satellite imagery long before capital equipment is placed into service.
Data in Practice: Seven-Layer Physical Infrastructure & Capex Architecture
Every record in the physical infrastructure feed is anchored to its publication timestamp and resolved to the corporate parent and ticker valid on that date, tracking 107 data-center sites (84 linked to a listed company across 16 tickers as of 2026-10-16). Across 454 air-quality permits (345 linked to a listed company, 76.0%, with 389 recording backup-generator MW values and 451 recording fuel type as of 2026-10-16), emergency-generator MW on an air permit is an upper bound on backup capacity at the site, not the data center's IT load. On the grid side, 101,852 substations (40,398 with a voltage value), 1,720 interconnection queue rows (723 linked to a listed company), and 9,639 power-plant capability rows (3,174 linked to a listed company) pair with 92,837 daily regional electricity demand rows across 14 grid balancing authorities from January 2019 (as of 2026-10-16).
To cross-reference physical site progress against supply-chain and balance-sheet commitments, the feed integrates 740 monthly US import rows by product code and origin country from January 2013, 2,812 quarterly company filing facts (CapEx, PP&E, Depreciation, ConstructionInProgress, Leases, and Commitments across 16 tickers from 2009), and 5,822 monthly public satellite surface indices across 95 sites from January 2018 (as of 2026-10-16). Note that monthly equipment imports by product code and origin country are industry-level context, not company-level shipments or sales. For a detailed empirical walkthrough, read our research briefing on AI Data Center Power Demand and our technical working paper Ground-Truthing the AI Capex Cycle: A Multi-Source Point-in-Time Dataset of Data Center Infrastructure, or inspect live row counts on Data Coverage & Completeness and Data Preview.
What related alternative intelligence streams are available?
Cross-reference correlated public records across our alternative intelligence directories, statutory trackers, and quantitative research libraries:
- Data Coverage & Completeness
- Data Preview
- Verification Methodology
- Track Record
- API for Funds & Fintech
- Federal contracts radar
- Congressional trading directory
- Government contractors directory
Access Real-Time Terminal Intelligence & Quantitative Signals
Unlock instant Telegram alerts, full congressional portfolio archives, and algorithmic catalyst radar.
Upgrade to Gemral Edge Pro ($39/mo)Frequently asked questions
What are Gemral Edge Public Datasets?
Gemral Edge Public Datasets are structured, standardized data archives published freely for academic researchers, quantitative developers, and open-source contributors. These datasets aggregate historical public disclosures—including congressional stock transactions, federal procurement contract obligations, executive guidance milestones, and sovereign central bank balance sheet time series—into clean, normalized CSV and JSON formats. By eliminating the technical friction of scraping and entity resolution, these public feeds enable researchers to backtest quantitative factors, train machine learning models, and analyze public capital flows with complete data provenance. All data signals on Gemral Edge are delivered with complete public provenance, maintaining strict non-partisan objectivity and transparent computational methodology without constituting investment advice. Every regulatory event retains its original statutory filing accession number, enabling compliance and investment teams to audit raw data directly. Historical time series are maintained with strict point-in-time veracity to prevent survivorship and look-ahead biases in quantitative research models.
What open licenses govern the public datasets?
Gemral Edge Public Datasets are released under open, transparent licensing frameworks designed to maximize academic and research utility while respecting public domain statutory data. Core public government records—such as federal contract data and STOCK Act filings—are published under Creative Commons Zero (CC0 1.0 Universal) dedication, ensuring unrestricted public domain access. Derived indices, standardized schemas, and normalized entity crosswalks are made available under Creative Commons Attribution (CC-BY 4.0), permitting open redistribution, adaptation, and commercial use with appropriate attribution to Gemral Edge. All data signals on Gemral Edge are delivered with complete public provenance, maintaining strict non-partisan objectivity and transparent computational methodology without constituting investment advice. Every regulatory event retains its original statutory filing accession number, enabling compliance and investment teams to audit raw data directly. Historical time series are maintained with strict point-in-time veracity to prevent survivorship and look-ahead biases in quantitative research models.
How can researchers and AI models access these datasets?
Researchers and autonomous AI models can access Gemral Edge datasets through direct public download links, GitHub repositories, Kaggle datasets, and native Model Context Protocol (MCP) server endpoints. Public CSV feeds are indexed with transparent column dictionaries and schema specifications, allowing seamless ingestion into Python pandas, R, and SQL databases. Autonomous AI assistants—such as Claude Code and ChatGPT—can query these datasets directly via declarative MCP tools, enabling programmatic synthesis of congressional trading data, contract velocity, and macro liquidity trends. All data signals on Gemral Edge are delivered with complete public provenance, maintaining strict non-partisan objectivity and transparent computational methodology without constituting investment advice. Every regulatory event retains its original statutory filing accession number, enabling compliance and investment teams to audit raw data directly. Historical time series are maintained with strict point-in-time veracity to prevent survivorship and look-ahead biases in quantitative research models.
How are proprietary metrics protected in public data exports?
Public dataset exports focus on factual statutory disclosures, normalized entity identifiers, and foundational time series, ensuring complete public availability of underlying government records. Real-time feeds and convergence alerts are part of the paid Pro and Premium plans, while programmatic access (the REST API v1 and HMAC-signed webhooks) is available only on the API plan. This dual model ensures open data access for researchers while sustaining institutional analytical infrastructure. All data signals on Gemral Edge are delivered with complete public provenance, maintaining strict non-partisan objectivity and transparent computational methodology without constituting investment advice. Every regulatory event retains its original statutory filing accession number, enabling compliance and investment teams to audit raw data directly. Historical time series are maintained with strict point-in-time veracity to prevent survivorship and look-ahead biases in quantitative research models. Platform data models follow strict entity resolution standards, linking obscure procurement subsidiaries and corporate divisions to primary trading tickers.
Risk Disclaimer
Trading and investing in digital assets, financial instruments, and predictive events involve substantial risk of loss and are not suitable for every investor. The predictive intelligence, probability distributions, historical precedents, and scenario modeling presented on this page are compiled for informational and research purposes only and do not constitute financial, investment, legal, or tax advice. Past performance and statistical precedents do not guarantee future outcomes. Always conduct independent due diligence before committing capital.