During the last months of 2020, Yields.io Team has been doubling efforts to improve Chiron. Curious? Scroll down to read more about the new features and enhancements.
New Features
Model and data set copy
Some of the objects one defines within Chiron may take a while to configure properly. As a consequence, being able to start from an already existing and correctly configured object to define a small alteration of it makes it easier for users to operate within the platform and create quicker new or enhanced pipelines.
This new ‘copy’ functionality hence allows one to create a new object by starting from an already existing object that is used as a ‘template’. This has been implemented for:
- model
- data set
Same functionality for ‘Data Model Mapping’ and ‘Instance’ shall follow in a next release.
New dimension ‘types’ in data model mapping
It has been noted that some of the dimensions contained in a dataset may not be directly related to the model one wants to analyze. Some of these dimensions may bring some additional context and may become handy to enrich model validation analysis. Others can be considered as completely ‘void’ for the model so that they shall not be loaded for model analysis. This is what we now allow with the new dimension types:
- Context: dimension is loaded in stage instances runs that will be using this DMM
- Excluded: dimension is not loaded in stage instances runs that will be using this DMM
Stage instance rerun with latest code
Following recommendations from our existing clients, some new functionalities have been added to enrich our instance objects lifecycle. The first one consists in being able to generate new sessions for an instance after adapting its code and despite the fact that no new data was added to the datasets the instance depends on.
Indeed, a stage instance is built on top of some template script. A copy of this template script is made when creating the instance so that user can adapt code for this particular instance if necessary. So far, session(s) for a given stage instance could only be created when new data was added after the latest session run to the datasets the stage instance depends on. Consequently, users had to delete the latest session in order to be able to rerun it with new code and without adding new data to the dataset.
From now on, new session(s) will be created automatically too when the script git-version has been incremented since latest session run.
Audit trail for DMM and stage instance
Following what had already been done for the model and data set objects, we have now added audit trail information for the data model mapping and instance object. This concludes a first implementation across the platform main objects of the ‘audit trail’ functionality.
Enhancements
Extended search functionality
The filtering functionality in our main objects views (i.e. model, data set, DMM, instance) has been improved to include more objects attributes. In the below example we show that filtering now operates on the ingested file names too for data sets objects.
Data set richer display and behavior
The data set object remains central to the Chiron application. Consequently, we keep working at making its lifecycle more easily manageable for the user. In this context, following improvements have been made around the data set object
- The card of a dataset in the “Ingested” status contains the last ingestion date, inspection report generation status, and the data file name
- Users are notified more explicitly about the ingestion report completion
- Users can write and dynamically adapt a description for the data set
DMM search functionality
Same datasets may have many dimensions. In this scope it is important to enable our users to easily navigate through these many dimensions when defining the model signature with a Data Model Mapping object. We thus added a first line in our DMM view that enables our users to more easily filter over the dimensions.
Stop all pending sessions
When running a ‘bacthkey’ type of stage instance many sessions may be created, i.e. one per observed value of the ‘batchkey’ dimension. It is sometimes the case that user realizes something is going wrong with such instances when the first session, out of these many created sessions, completes with a failure. In such a case, user may want to stop other pending sessions in order to fix the issue before proceeding further. This is now allowed by the ‘Stop all pending sessions’ button that appears in the particular instance view (see below screenshot)
IE and Edge compatibility
Chiron portal has been adapted to support IE11 and Edge. You should now be able to access our platform through these web browsers.
Infrastructure
YARN has now been introduced for better computation resources management. Both our data-lake as well as our session runner engines can now leverage this new setup.
Conclusion
This is it for now! If you want to share any feedback with us, contact us
Interested in learning more? Watch a demo of Chiron, our flagship product.
