diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 72dbea0..8b0613c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,3 +1,17 @@ +Content + +* [code base](#code-base) +* [config parsing](#config-parsing) + * [lower layer](#lower-layer) + * [higher layer](#higher-layer) + * [Precedence](#precedence) + * [variables resolution](#variables-resolution) + * [rules](#rules) +* [testing](#testing) + * [testing with unittest](#testing-with-unittest) + * [testing with bash scripts](#testing-with-bash-scripts) +* [documentation](#documentation) + Thanks for helping out! Feature requests, bug reports and PRs are always welcome! @@ -6,9 +20,9 @@ This file provides a few pointers on how to contribute to dotdrop and where to find information. For any question, feel free to open an issue. For PR adding new features, I'd be very thankful if you could add either -a unittest testing the added feature or a bash script test, thanks! +a unittest testing the added feature or a bash script test ((see [testing](#testing), thanks! -# Code base +# code base Dotdrop's code base is located in the [dotdrop directory](/dotdrop). @@ -32,25 +46,100 @@ Here's an overview of the different files and their role: * **updater.py**: the class handling the update of dotfiles for `update` * **utils.py**: some useful methods -## Config parsing +# config parsing -The configuration file (yaml) is parsed in two layers: +The configuration file (yaml) is parsed using two layers: - * the lower layer in `cfg_yaml.py` - * the higher layer in `cfg_aggregator.py` + * first in the lower layer in [cfg_yaml.py](/dotdrop/cfg_yaml.py) + * then in the higher layer in [cfg_aggregator.py](/dotdrop/cfg_aggregator.py) Only the higher layer is accessible to other classes of dotdrop. -**Rules** +## lower layer + +This is done in [cfg_yaml.py](/dotdrop/cfg_yaml.py) + +The lower layer part is only taking care of basic types +and does the following: + * normalize all config entries + * resolve paths (dotfiles src, dotpath, etc) + * refactor actions/transformations to a common format + * etc + * import any data from external files (configs, variables, etc) + * apply variable substitutions + * complete any data if needed (add the "profile" variable, etc) + * execute intrepreted variables through the shell + * write new entries (dotfile, profile) into the dictionary and save it to a file + * fix any deprecated entries (link_by_default, etc) + * clear empty entries + +In the end it builds a cleaned and normalized dictionary to be accessed by the higher layer. + +## higher layer + +This is done in [cfg_aggregator.py](/dotdrop/cfg_aggregator.py) + +The higher layer will transform the dictionary parsed by the lower layer +into objects (profiles, dotfiles, actions, transformations, etc). +The higher layer has no notion of inclusion (profile included for example) or +file importing (import actions, etc) or even interpreted variables +(it only sees variables that have already been interpreted). + +It does the following: + * transform dictionaries into objects + * patch list of keys with its corresponding object (for example dotfile's actions) + * provide getters for every other classes of dotdrop needing to access elements + +Note that any change to the yaml dictionary (adding a new profile or a new dotfile for +example) won't be *seen* by the higher layer until the config is reloaded. Consider the +`dirty` flag as a sign the file needs to be written and its representation in higher +levels in not accurate anymore. + +## precedence + +* `dynvariables` > `variables` +* profile `(dyn)variables` > any other `(dyn)variables` +* profile `(dyn)variables` > profile's included `(dyn)variables` +* imported `variables`/`dynvariables` > `(dyn)variables` + +## variables resolution + +How variables are resolved (through jinja2's +templating) in the config file. + +* resolve main config file variables + * merge `variables` and `dynvariables` (allowing cycling reference) + * recursively template merged `variables` and `dynvariables` + * `dynvariables` are executed + * profile's `variables` and `dynvariables` are merged +* resolve *included* entries (see below) + * paths and entries are templated + (allows to use something like `include {{@@ os @@}}.variables.yaml`) +* *included* entries are processed + * dyn-/variables are all resolved in their own file + +potential *included* entries + +* entry *import_actions* +* entry *import_configs* +* entry *import_variables* +* profile's *import* +* profile's *include + +Variables are then used to resolve different elements in the config file: +see [this](https://github.com/deadc0de6/dotdrop/wiki/config-variables#config-available-variables) + +## rules * `dynvariables` are executed in their own config file * since `variables` and `dynvariables` are templated before the `dynvariables` are executed, this means that `dynvariables` can safely reference `variables` however - `variables` referencing `dynvariables` will result with the none executed value of the + `variables` referencing `dynvariables` will result with the *not-executed* value of the referenced `dynvariables` (see examples below) * profile cannot include profiles defined above in the import tree -* config files do not have access to variables - defined above in the import tree +* config files do not have access to variables defined above in the import tree +* actions/transformations using variables are resolved at runtime + (when action/transformation is executed) and not when loading the config This will result with `dvar0 = "test"` and `var0 = "echo test"` (**not** `var0 = test`) ```yaml @@ -68,80 +157,8 @@ dynvariables: dvar0: "echo {{@@ var0 @@}}" ``` -**Precedence** -* `dynvariables` > `variables` -* profile `(dyn)variables` > any other `(dyn)variables` -* profile `(dyn)variables` > profile's included `(dyn)variables` -* imported `variables`/`dynvariables` > `(dyn)variables` -* actions/transformations using variables are resolved at runtime - (when action/transformation is executed) and not when loading the config - -## lower layer (cfg_yaml.py) - -The lower layer part is only taking care of basic types -does the following: - * normalize all config entries - * resolve paths (dotfiles src, dotpath, etc) - * refactor actions to a common format - * etc - * import any data from external files (configs, variables, etc) - * apply variable substitutions - * complete any data if needed (add the "profile" variable, etc) - * execute intrepreted variables through the shell - * write new entries (dotfile, profile) into the dictionary and save it to a file - * fix any deprecated entries (link_by_default, etc) - * clear empty entries - -In the end it makes sure the dictionary accessed -by the higher layer is clean and normalized. - -## higher layer (cfg_aggregator.py) - -The higher layer will transform the dictionary parsed by the lower layer -into objects (profiles, dotfiles, actions, etc). -The higher layer has no notion of inclusion (profile included for example) or -file importing (import actions, etc) or even interpreted variables -(it only sees variables that have already been interpreted). - -It does the following: - * transform dictionaries into objects - * patch list of keys with its corresponding object (for example dotfile's actions) - * provide getters for every other classes of dotdrop needing to access elements - -Note that any change to the yaml dictionary (adding a new profile or a new dotfile for -example) won't be *seen* by the higher layer until the config is reloaded. Consider the -`dirty` flag as a sign the file needs to be written and its representation in higher -levels in not accurate anymore. - -## Variables resolution in the config file - -How variables are resolved (through jinja2's -templating) in the config file. - -* resolve main config file variables - * merge `variables` and `dynvariables` (allowing cycling reference) - * recursively template merged `variables` and `dynvariables` - * `dynvariables` are executed - * profile's `variables` and `dynvariables` are merged -* resolve *included* entries (see below) - * paths and entries are templated - (allows to use something like `include {{@@ os @@}}.variables.yaml`) -* *included* entries are processed - * dyn-/variables are all resolved in their own file - -*included* - -* entry *import_actions* -* entry *import_configs* -* entry *import_variables* -* profile's *import* -* profile's *include - -Variables are then used to resolve different elements in the config file: -see [this](https://github.com/deadc0de6/dotdrop/wiki/config-variables#config-available-variables) - -# Testing +# testing Dotdrop is tested with the use of the [tests.sh](/tests.sh) script. @@ -166,6 +183,6 @@ for different use-cases (usually described in their filename). Each script starts with the same boiler plate code that you can paste at the start of your new test (see the head of the file down to `# this is the test`). -# Documentation +# documentation Most of dotdrop documentation is hosted in [its wiki](https://github.com/deadc0de6/dotdrop/wiki)