ARK 0: A Quarter-Life Crisis?

Gary Davies

Do you not mean Mid-Life Crisis?

Hold on a second, I’m not quite there yet, I’m talking about kdb+. Did you know kdb reached a significant milestone in 2023 hitting 25 years, a quarter of a century, 20 of which as kdb+?

What is a quarter-life crisis? According to wikipedia, it’s a crisis involving “anxiety over the direction and quality of ones life”.

I’d suggest kdb+ as a technology has already moved past its quarter-life crisis, it has clear direction with the addition of some excellent new KX additions over the last few years:

KX Insights / Microservices
PyKX
PGWire
kdb.ai
Cloud compatibility e.g. AWS Finspace

These additions, combined with kdb+’s raw power, enhance the quality of time series data and vector analytics solutions.

Although, kdb+ as a technology may be through it, I’d argue that applications and implementations utilizing kdb+ might still face a crisis of direction and quality. The applications in question aren’t necessarily 18-25 years in production either, but that would make for a less click-bait blog title.

kdb+ applications face the challenge of delivering high-speed data capture and analytics in a dynamic and fast paced environment, with ever changing demand for more data, more users and more functionality. Maintaining the quality and the direction of the application can be difficult, especially if application issues or new functionality hinder our ability to review the application as a whole.

In 2023, Data Intellect launched our ARK program (Application Review for kdb+) which encompasses our kdb+ maturity matrix (the secret sauce).

This blog, Episode 0, marks the beginning of a blog series focusing on ARK.

What is ARK?

ARK is an offering from Data Intellect where we will provide an independent architecture review of any kdb+ system, making use of our depth and breadth of knowledge of systems and challenges across the street.

Our subject matter experts will leverage this experience to provide an assessment of the kdb+ system, based on its current and/or projected requirements, to identify opportunities for improvement.

The review utilizes our kdb+ maturity matrix which provides a quantitative scoring across 12 key areas:

kdb+ framework
Hardware
Environments
Disaster Recovery / Continuity of Business
Software Development Life Cycle
Performance & Scalability
Security & Access Controls
Supportability
Test-ability
Data Quality
Data Availability
Code

This review can also encapsulate application interoperability with the new KX product line if your focus is about getting to the cloud or integrating with AI.

Clients are presented with a detailed report of scoring, findings and any actions that must, should or could be actioned upon to improve the system:

Changes to meet current/increased demand
Design opportunities, leveraging the latest developments in the kdb+ space, e.g kdb.ai, PyKX
Improvements for system throughput
System simplifications
Performance improvements to meet current or improved Service Level Agreements (SLAs)
Cloud Architecture and migration pathways
kdb+ Upgrades
Improvements to development pipelines
System Stability

Factor of 12

In line with kdb+, and the correct vector notation, this is the 0th blog – the purpose of introducing ARK and what is to come.

Over the course of the next 12 months, on a monthly basis, a Data Intellect subject matter expert is going to produce a blog corresponding to a maturity characteristic.

Each blog will be their own thoughts on the area.

12 months, 12 maturity characteristics, 12 subject matter experts