During my career I have worked as both a Software Developer and Support Engineer. Over the years, I have learned that Geneos Active Console, part of the suite of monitoring tools built by ITRS, is an immensely powerful tool for supporting kdb+ applications. In my opinion, it is by far the best tool for monitoring large kdb+ frameworks. This is primarily due to its layered approach and the dynamic functionality that can be built as a result. Geneos gives developers and support engineers a structured interface, a highly configurable layout and intuitive alerting for their systems. The larger your kdb+ application grows, the more invaluable Geneos becomes.
The software does come with an initial learning curve for new users. I’ve witnessed, at least in the kdb+ world, a relatively long setup time as developers and support wrestle with countless other tasks before an application goes live. This can lead to a headache for management – who builds the monitoring on Geneos?
From my personal experiences, the bulk of Geneos knowledge frequently lies with the Support or Operations teams. Conversely, the majority of kdb+ expertise and knowledge of the application’s upcoming features belongs to the developers. This is particularly true in institutions where the Support teams work across several applications, in multiple different programming languages. So unless every institution is willing to hire a brand new team of Geneos savvy, kdb+ trained Support Analysts (if you are, then please get in touch), then setting up an all-encompassing Geneos monitoring system is likely going to be a time-consuming and complicated endeavour.
At least, until now…
AquaQ Analytics have created a kdb+/Geneos integrated framework. This will allow users to programatically generate a Geneos Active Console XML for their application. Instead of creating each Managed Entity, Type, Sampler, Rule, Active Time and Process Descriptor separately, the user can populate a few simple csv files with variables and q queries and a script will populate and import the output to a Geneos includes file. No prior knowledge of Geneos or the ability to write in XML is required. The package comes with several predefined samplers. These should be useful in looking at time series data in a kdb+ application. While originally designed on a TorQ framework, the Qeneos XML builder comes with instructions on how to configure it for use with other frameworks.
In our first version we prioritised the basics. The initial goal was to create a product as close to plug-and-play as possible, with a priority on adding value to any user in the minimum time possible. Releasing this initial version gives us the opportunity to listen to anyone interested in the framework, and we can then continue our development to meet users needs.
Version 1 of Qeneos boasts the following features:
While everything is configurable and you can structure your Geneos however you see fit, we have created the recommended model below:
One of the simplest, but most powerful parts of the XML builder framework is the ability to turn a kdb+ query or function into a geneos sampler. Anything that can be coded to return a table can be added to your Geneos.
We achieve this with a combination of q scripts. The first is designed to isolate the process you’re trying to connect to and return the connection details. The second is programmed to run your q query against the particular process and convert it into a Geneos-readable output. This is neatly wrapped in a single bash script, allowing it to be easily configured into existing monitoring systems and the TorQ framework.
From years of experience and collaboration, I have outlined my personal methodology and approach to building a Geneos monitoring system for a kdb+ application:
Building Qeneos version one has been a lot of fun and a great learning experience for us at AquaQ. We’re hopeful that this will prove a useful tool to Geneos and kdb+ users and that we can continue developing the product. The enhancements on the horizon include Commands, Actions, Authentication and Exported Data, as well as integrating it with some new monitoring features we are developing for AquaQ’s TorQ framework.
Ideally though we want to be building things that benefit you! If you have any feature requests, have found bugs or just fancy a chat and some more information about supporting your kdb+ application with Geneos Active Console, please bombard us with emails, comment on our social media or get our attention in any way you see fit.
If your organisation has access to Geneos then please try downloading Qeneos and giving it a try!
AquaQ are always looking for talented individuals to work in both their support or development streams. Please check out our available opportunities if you’re interested in joining the team.