DataOps

DataOps in Glean allows you to represent your resources as configuration files, which can be stored in a Git repository and deployed to your Glean project as part of a change management workflow.

Using DataOps, you can:

• ✅ Validate planned changes to your data warehouse or Glean resources to ensure that your views and dashboards don't break
• 🏗️ Preview updates to your Glean project before making them visible to the rest of your organization
• 👥 Use code reviews or pull requests to collaborate on proposed changes
• 🧑‍💻 Configure Glean using the same tools that you use to develop your backend pipelines
• 🚦 Use a continuous integration system to deploy updates to your Glean project

DataOps is under active development and currently has some limitations, including:

• The Glean configuration files do not yet support every feature that is available in the Glean web application.
• If you build resources from a Git repository, users are still able to edit those resources through the Glean UI. When you re-deploy these resources with your DataOps configuration files, any changes made through the Glean UI will be overwritten.

DataOps Basics

To use DataOps, you define one or more of your Glean resources using configuration files. A configuration file contains a complete specification of a resource. For instance, a Model configuration file contains information about which database connection to use, the name of the underlying database table, a list of attributes and metrics, etc.

Using a set of configuration files, you create a Build. There are two different kinds of Builds:

• A Preview Build validates your configuration files and, if successful, provides a URL that will show you what your Glean Project will look like if your pending changes are applied.
• A Deploy Build validates your configuration files and, if successful, publishes those changes to your Glean Project.