In 2024, Data Intellect launched a dedicated Databricks Services Practice, driven by the transformative impact of Databricks technology seen by clients. Leveraging over 15 years of our experience in Time Series analysis, Front Office Data, and Trading Technology, we bring unparalleled new expertise to Financial Services and Capital Markets.
By combining our deep industry knowledge with the powerful Databricks Technology & Partner Ecosystem, we deliver world-class solutions designed to unlock value and drive innovation for your business.
Bringing technology, industry and use case expertise, Data Intellect help you make the most of the Databricks Lakehouse Platform. Services include designing a data transformation strategy unique to your business, data modernisation and migration, data management and governance, and much more.
By leveraging the lightning-fast performance of kdb+ and KX's Python library (PyKX), Databricks users can now enhance their data-driven models directly within their Databricks environment via KX's powerful columnar and functional programming capability, without requiring q language expertise. This introduces Databricks to Capital Markets Front Office conversations.
Data Intellect are expert in both technologies and can guide project implementations.
Databricks illustrates how the timestamp key is used to ensure point-in-time correctness in a training dataset. Feature values are matched based on the primary key (not shown in the diagram) and the timestamp key, using an AS OF join. The AS OF join ensures that the most recent value of the feature at the time of the timestamp is used in the training set.
Data Intellect are expert in Time Series Technologies and can guide project implementations.