Managing Resources With Code
Here is a high-level description of what actually happens during the build process:
- Create resources. For models that you check in that do not currently exist, the model is created from scratch from your configuration file and from the database. This means that the base table and all the columns must still exist in the database for the build to succeed.
- Update resources. If a column is changed, it will be updated in all dependent items (regardless of whether the dependent items were also checked in) - so things like name changes should be correctly reflected.
- Clean up resources. If any columns or models are deleted, all dependent items that relied on that column or model will also be deleted.
- Smoke test. Once the model is built, a single aggregate analytical query will be run against the database to make sure that data aggregation can still be run against the model.
If there is an error while deploying a Data Model, the build process will revert to the previous version of that model, and all dependent resources of that model will not be updated.
Glean supports configuration files written in YAML or JSON. A Glean Build will attempt to validate every file in the specified path with a
Full documentation of all configuration files can be found for each resource type:
An easy way to get started building a configuration file is to export it from an existing Glean resource.
- Data Models:
- Navigate to the Data Models (opens in a new tab) page
Editfor one of your Data Models
- Click the
⋮in the top-right corner
- Saved Explorations: On any Explore page, click the
⋮in the top-right corner of the chart area, and then click
Export Saved Exploration Config File.
- Dashboards: On any Dashboard page, click the
⋮in the top-right corner, and then click
View DataOps Config File.
If you have existing content built in the UI and want to move it into DataOps,
you can specify the
grn property on the resource. This will cause the config to
update an existing resource with that GRN, instead of creating a
This can also be used to avoid recreating resources when renaming or refactoring the file structure of your project.
To protect specific resources to only be editable with DataOps, add the
preventUpdatesFromUI flag set as
true to any config. This prevents users from unintentionally updating critical resources from the Glean web application and gives your team greater control over valuable long lived resources that are core to your business needs.