Merge pull request #257 from deadc0de6/bug250

Bug250

CONTRIBUTING.md

@@ -1,3 +1,17 @@
+Content
+
+* [code base](#code-base)
+* [config parsing](#config-parsing)
+  * [lower layer](#lower-layer)
+  * [higher layer](#higher-layer)
+  * [Precedence](#precedence)
+  * [variables resolution](#variables-resolution)
+  * [rules](#rules)
+* [testing](#testing)
+  * [testing with unittest](#testing-with-unittest)
+  * [testing with bash scripts](#testing-with-bash-scripts)
+* [documentation](#documentation)
+
 Thanks for helping out!
 
 Feature requests, bug reports and PRs are always welcome!
@@ -6,9 +20,9 @@ This file provides a few pointers on how to contribute to dotdrop
 and where to find information. For any question, feel free to open an issue.
 
 For PR adding new features, I'd be very thankful if you could add either
-a unittest testing the added feature or a bash script test, thanks!
+a unittest testing the added feature or a bash script test ((see [testing](#testing), thanks!
 
-# Code base
+# code base
 
 Dotdrop's code base is located in the [dotdrop directory](/dotdrop).
 
@@ -32,20 +46,24 @@ Here's an overview of the different files and their role:
 * **updater.py**: the class handling the update of dotfiles for `update`
 * **utils.py**: some useful methods
 
-## Config parsing
+# config parsing
 
-The configuration file (yaml) is parsed in two layers:
+The configuration file (yaml) is parsed using two layers:
 
-  * the lower layer in `cfg_yaml.py`
-  * the higher layer in `cfg_aggregator.py`
+  * first in the lower layer in [cfg_yaml.py](/dotdrop/cfg_yaml.py)
+  * then in the higher layer in [cfg_aggregator.py](/dotdrop/cfg_aggregator.py)
 
 Only the higher layer is accessible to other classes of dotdrop.
 
-The lower layer part is only taking care of basic types and
-does the following:
+## lower layer
+
+This is done in [cfg_yaml.py](/dotdrop/cfg_yaml.py)
+
+The lower layer part is only taking care of basic types
+and does the following:
   * normalize all config entries
     * resolve paths (dotfiles src, dotpath, etc)
-    * refactor actions to a common format
+    * refactor actions/transformations to a common format
     * etc
   * import any data from external files (configs, variables, etc)
   * apply variable substitutions
@@ -55,11 +73,14 @@ does the following:
   * fix any deprecated entries (link_by_default, etc)
   * clear empty entries
 
-In the end it makes sure the dictionary (or parts of it) accessed
-by the higher layer is clean and normalized.
+In the end it builds a cleaned and normalized dictionary to be accessed by the higher layer.
+
+## higher layer
+
+This is done in [cfg_aggregator.py](/dotdrop/cfg_aggregator.py)
 
 The higher layer will transform the dictionary parsed by the lower layer
-into objects (profiles, dotfiles, actions, etc).
+into objects (profiles, dotfiles, actions, transformations, etc).
 The higher layer has no notion of inclusion (profile included for example) or
 file importing (import actions, etc) or even interpreted variables
 (it only sees variables that have already been interpreted).
@@ -74,37 +95,70 @@ example) won't be *seen* by the higher layer until the config is reloaded. Consi
 `dirty` flag as a sign the file needs to be written and its representation in higher
 levels in not accurate anymore.
 
-## Variables resolution
-
-How variables are resolved (pass through jinja2's
-templating function) in the config file.
-
-* resolve `include` (the below merge is temporary just to resolve the `includes`)
-  * `variables` and `dynvariables` are first merged and recursively resolved
-  * `dynvariables` are executed
-  * they are all merged and `include` paths are resolved
-    (allows to use something like `include {{@@ os @@}}.variables.yaml`)
-* `variables` and profile's `variables` are merged
-* `dynvariables` and profile's `dynvariables` are merged
-* `dynvariables` are executed
-* they are all merged into the final *local* `variables`
-
-These are then used to resolve different elements in the config file:
-see [this](https://github.com/deadc0de6/dotdrop/wiki/config-variables#config-available-variables)
-
-Then additional variables (`import_variables` and `import_configs`) are
-then merged and take precedence over local variables.
-
-Note:
+## precedence
 
 * `dynvariables` > `variables`
 * profile `(dyn)variables` > any other `(dyn)variables`
 * profile `(dyn)variables` > profile's included `(dyn)variables`
 * imported `variables`/`dynvariables` > `(dyn)variables`
-* actions/transformations  using variables are resolved at runtime
+
+## variables resolution
+
+How variables are resolved (through jinja2's
+templating) in the config file.
+
+* resolve main config file variables
+  * merge `variables` and `dynvariables` (allowing cycling reference)
+  * recursively template merged `variables` and `dynvariables`
+  * `dynvariables` are executed
+  * profile's `variables` and `dynvariables` are merged
+* resolve *included* entries (see below)
+  * paths and entries are templated
+    (allows to use something like `include {{@@ os @@}}.variables.yaml`)
+* *included* entries are processed
+  * dyn-/variables are all resolved in their own file
+
+potential *included* entries
+
+* entry *import_actions*
+* entry *import_configs*
+* entry *import_variables*
+* profile's *import*
+* profile's *include
+
+Variables are then used to resolve different elements in the config file:
+see [this](https://github.com/deadc0de6/dotdrop/wiki/config-variables#config-available-variables)
+
+## rules
+
+* `dynvariables` are executed in their own config file
+* since `variables` and `dynvariables` are templated before the `dynvariables`
+  are executed, this means that `dynvariables` can safely reference `variables` however
+  `variables` referencing `dynvariables` will result with the *not-executed* value of the
+  referenced `dynvariables` (see examples below)
+* profile cannot include profiles defined above in the import tree
+* config files do not have access to variables defined above in the import tree
+* actions/transformations using variables are resolved at runtime
   (when action/transformation is executed) and not when loading the config
 
-# Testing
+This will result with `dvar0 = "test"` and `var0 = "echo test"` (**not** `var0 = test`)
+```yaml
+variables:
+  var0: "{{@@ dvar0 @@}}"
+dynvariables:
+  dvar0: "echo test"
+```
+
+This will result with `dvar0 = "test"` and `var0 = "test"`
+```yaml
+variables:
+  var0: "test"
+dynvariables:
+  dvar0: "echo {{@@ var0 @@}}"
+```
+
+
+# testing
 
 Dotdrop is tested with the use of the [tests.sh](/tests.sh) script.
 
@@ -129,6 +183,6 @@ for different use-cases (usually described in their filename).
 Each script starts with the same boiler plate code that you can paste at the
 start of your new test (see the head of the file down to `# this is the test`).
 
-# Documentation
+# documentation
 
 Most of dotdrop documentation is hosted in [its wiki](https://github.com/deadc0de6/dotdrop/wiki)

README.md

@@ -38,7 +38,7 @@ Features:
 
 Check also the [blog post](https://deadc0de.re/articles/dotfiles.html),
 the [example](#getting-started), the [wiki](https://github.com/deadc0de6/dotdrop/wiki) or
-how [people are using dotdrop](https://github.com/deadc0de6/dotdrop/wiki/people-using-dotdrop)
+how [people are using dotdrop](https://github.com/deadc0de6/dotdrop/wiki/meta-people-using-dotdrop)
 for more.
 
 Quick start:
@@ -261,8 +261,8 @@ That's it, a single repository with all your dotfiles for your different hosts.
 
 You can then
 
-* [create actions](https://github.com/deadc0de6/dotdrop/wiki/usage-actions)
-* [use transformations](https://github.com/deadc0de6/dotdrop/wiki/usage-transformations)
+* [create actions](https://github.com/deadc0de6/dotdrop/wiki/config-actions)
+* [use transformations](https://github.com/deadc0de6/dotdrop/wiki/config-transformations)
 * [use variables](https://github.com/deadc0de6/dotdrop/wiki/config-variables)
 * [symlink dotfiles](https://github.com/deadc0de6/dotdrop/wiki/symlinked-dotfiles)
 * [and more](https://github.com/deadc0de6/dotdrop/wiki)

dotdrop/action.py

@@ -11,6 +11,7 @@ import os
 
 # local imports
 from dotdrop.dictparser import DictParser
+from dotdrop.exceptions import UndefinedException
 
 
 class Cmd(DictParser):
@@ -32,7 +33,12 @@ class Cmd(DictParser):
         ret = 1
         action = self.action
         if templater:
-            action = templater.generate_string(self.action)
+            try:
+                action = templater.generate_string(self.action)
+            except UndefinedException as e:
+                err = 'bad {}: {}'.format(self.descr, e)
+                self.log.warn(err)
+                return False
             if debug:
                 self.log.dbg('{}:'.format(self.descr))
                 self.log.dbg('  - raw       \"{}\"'.format(self.action))
@@ -42,7 +48,12 @@ class Cmd(DictParser):
         if self.args:
             args = self.args
             if templater:
-                args = [templater.generate_string(a) for a in args]
+                try:
+                    args = [templater.generate_string(a) for a in args]
+                except UndefinedException as e:
+                    err = 'bad arguments for {}: {}'.format(self.descr, e)
+                    self.log.warn(err)
+                    return False
         if debug and args:
             self.log.dbg('action args:')
             for cnt, arg in enumerate(args):

dotdrop/cfg_aggregator.py

@@ -17,6 +17,7 @@ from dotdrop.profile import Profile
 from dotdrop.action import Action, Transform
 from dotdrop.logger import Logger
 from dotdrop.utils import strip_home
+from dotdrop.exceptions import UndefinedException
 
 
 TILD = '~'
@@ -77,7 +78,7 @@ class CfgAggregator:
             self._debug_list('trans_w', self.trans_w)
 
         # variables
-        self.variables = self.cfgyaml.get_variables()
+        self.variables = self.cfgyaml.variables
         if self.debug:
             self._debug_dict('variables', self.variables)
 
@@ -134,8 +135,9 @@ class CfgAggregator:
                 objects.append(o)
             if not islist:
                 objects = objects[0]
-            if self.debug:
-                self.log.dbg('patching {}.{} with {}'.format(c, keys, objects))
+            # if self.debug:
+            #     er = 'patching {}.{} with {}'
+            #     self.log.dbg(er.format(c, keys, objects))
             setattr(c, keys, objects)
 
     def del_dotfile(self, dotfile):
@@ -281,7 +283,12 @@ class CfgAggregator:
         @src: dotfile src (in dotpath)
         @dst: dotfile dst (on filesystem)
         """
-        src = self.cfgyaml.resolve_dotfile_src(src)
+        try:
+            src = self.cfgyaml.resolve_dotfile_src(src)
+        except UndefinedException as e:
+            err = 'unable to resolve {}: {}'
+            self.log.err(err.format(src, e))
+            return None
         dotfiles = self.get_dotfile_by_dst(dst)
         for d in dotfiles:
             if d.src == src:

dotdrop/dotdrop.py

@@ -20,7 +20,7 @@ from dotdrop.comparator import Comparator
 from dotdrop.utils import get_tmpdir, remove, strip_home, \
     run, uniq_list, patch_ignores, dependencies_met
 from dotdrop.linktypes import LinkTypes
-from dotdrop.exceptions import YamlException
+from dotdrop.exceptions import YamlException, UndefinedException
 
 LOG = Logger()
 TRANS_SUFFIX = 'trans'
@@ -230,6 +230,7 @@ def cmd_compare(o, tmp):
         newvars = dotfile.get_dotfile_variables()
         t.add_tmp_vars(newvars=newvars)
 
+        # dotfiles does not exist / not installed
         if o.debug:
             LOG.dbg('comparing {}'.format(dotfile))
         src = dotfile.src
@@ -239,9 +240,9 @@ def cmd_compare(o, tmp):
             same = False
             continue
 
+        # apply transformation
         tmpsrc = None
         if dotfile.trans_r:
-            # apply transformation
             if o.debug:
                 LOG.dbg('applying transformation before comparing')
             tmpsrc = apply_trans(o.dotpath, dotfile, t, debug=o.debug)
@@ -261,20 +262,26 @@ def cmd_compare(o, tmp):
                 LOG.dbg('points to itself')
             continue
 
-        # install dotfile to temporary dir
-        ret, insttmp = inst.install_to_temp(t, tmp, src, dotfile.dst)
+        # install dotfile to temporary dir and compare
+        ret, err, insttmp = inst.install_to_temp(t, tmp, src, dotfile.dst)
         if not ret:
             # failed to install to tmp
+            line = '=> compare {}: error'
+            LOG.log(line.format(dotfile.key, err))
+            LOG.err(err)
             same = False
             continue
         ignores = list(set(o.compare_ignore + dotfile.cmpignore))
         ignores = patch_ignores(ignores, dotfile.dst, debug=o.debug)
         diff = comp.compare(insttmp, dotfile.dst, ignore=ignores)
+
+        # clean tmp transformed dotfile if any
         if tmpsrc:
-            # clean tmp transformed dotfile if any
             tmpsrc = os.path.join(o.dotpath, tmpsrc)
             if os.path.exists(tmpsrc):
                 remove(tmpsrc)
+
+        # print diff result
         if diff == '':
             if o.debug:
                 line = '=> compare {}: diffing with \"{}\"'
@@ -655,7 +662,10 @@ def main():
     try:
         o = Options()
     except YamlException as e:
-        LOG.err('config file error: {}'.format(str(e)))
+        LOG.err('config error: {}'.format(str(e)))
+        return False
+    except UndefinedException as e:
+        LOG.err('config error: {}'.format(str(e)))
         return False
 
     if o.debug:

dotdrop/exceptions.py

@@ -9,3 +9,8 @@ diverse exceptions
 class YamlException(Exception):
     """exception in CfgYaml"""
     pass
+
+
+class UndefinedException(Exception):
+    """exception in templating"""
+    pass

dotdrop/installer.py

@@ -12,6 +12,7 @@ import errno
 from dotdrop.logger import Logger
 from dotdrop.templategen import Templategen
 import dotdrop.utils as utils
+from dotdrop.exceptions import UndefinedException
 
 
 class Installer:
@@ -239,7 +240,6 @@ class Installer:
                 actionexec = None
             else:
                 if err:
-                    return ret, err
                     return self._log_install(ret, err)
 
         return self._log_install(installed > 0, None)
@@ -325,8 +325,12 @@ class Installer:
             err = 'dotfile points to itself: {}'.format(dst)
             return False, err
         saved = templater.add_tmp_vars(self._get_tmp_file_vars(src, dst))
-        content = templater.generate(src)
-        templater.restore_vars(saved)
+        try:
+            content = templater.generate(src)
+        except UndefinedException as e:
+            return False, str(e)
+        finally:
+            templater.restore_vars(saved)
         if noempty and utils.content_empty(content):
             if self.debug:
                 self.log.dbg('ignoring empty template: {}'.format(src))
@@ -547,9 +551,10 @@ class Installer:
         src = os.path.expanduser(src)
         dst = os.path.expanduser(dst)
         if self.debug:
-            self.log.dbg('tmp install {} to {}'.format(src, dst))
+            self.log.dbg('tmp install {} (defined dst: {})'.format(src, dst))
         # install the dotfile to a temp directory for comparing
-        ret, tmpdst = self._install_to_temp(templater, src, dst, tmpdir)
+        r, tmpdst = self._install_to_temp(templater, src, dst, tmpdir)
+        ret, err = r
         if self.debug:
             self.log.dbg('tmp installed in {}'.format(tmpdst))
         # reset flags
@@ -557,4 +562,4 @@ class Installer:
         self.diff = diffsaved
         self.comparing = False
         self.create = createsaved
-        return ret, tmpdst
+        return ret, err, tmpdst

dotdrop/options.py

@@ -109,9 +109,11 @@ class Options(AttrMonitor):
         """constructor
         @args: argument dictionary (if None use sys)
         """
-        self.args = args
+        self.args = {}
         if not args:
             self.args = docopt(USAGE, version=VERSION)
+        if args:
+            self.args = args.copy()
         self.log = Logger()
         self.debug = self.args['--verbose'] or ENV_DEBUG in os.environ
         self.dry = self.args['--dry']
@@ -122,6 +124,7 @@ class Options(AttrMonitor):
         self.confpath = self._get_config_path()
         if self.debug:
             self.log.dbg('version: {}'.format(VERSION))
+            self.log.dbg('command: {}'.format(' '.join(sys.argv)))
             self.log.dbg('config file: {}'.format(self.confpath))
 
         self._read_config()

dotdrop/templategen.py

@@ -7,12 +7,16 @@ jinja2 template generator
 
 import os
 from jinja2 import Environment, FileSystemLoader, \
-    ChoiceLoader, FunctionLoader, TemplateNotFound
+    ChoiceLoader, FunctionLoader, TemplateNotFound, \
+    StrictUndefined
+from jinja2.exceptions import UndefinedError
+
 
 # local imports
 import dotdrop.utils as utils
 from dotdrop.logger import Logger
 import dotdrop.jhelpers as jhelpers
+from dotdrop.exceptions import UndefinedException
 
 BLOCK_START = '{%@@'
 BLOCK_END = '@@%}'
@@ -36,6 +40,7 @@ class Templategen:
         self.base = base.rstrip(os.sep)
         self.debug = debug
         self.log = Logger()
+        self.variables = {}
         loader1 = FileSystemLoader(self.base)
         loader2 = FunctionLoader(self._template_loader)
         loader = ChoiceLoader([loader1, loader2])
@@ -47,11 +52,14 @@ class Templategen:
                                variable_start_string=VAR_START,
                                variable_end_string=VAR_END,
                                comment_start_string=COMMENT_START,
-                               comment_end_string=COMMENT_END)
+                               comment_end_string=COMMENT_END,
+                               undefined=StrictUndefined)
+
         # adding variables
-        self.env.globals['env'] = os.environ
+        self.variables['env'] = os.environ
         if variables:
-            self.env.globals.update(variables)
+            self.variables.update(variables)
+
         # adding header method
         self.env.globals['header'] = self._header
         # adding helper methods
@@ -72,32 +80,48 @@ class Templategen:
             self._debug_dict('template additional variables', variables)
 
     def generate(self, src):
-        """render template from path"""
+        """
+        render template from path
+        may raise a UndefinedException
+        in case a variable is undefined
+        """
         if not os.path.exists(src):
             return ''
-        return self._handle_file(src)
+        try:
+            return self._handle_file(src)
+        except UndefinedError as e:
+            err = 'undefined variable: {}'.format(e.message)
+            raise UndefinedException(err)
 
     def generate_string(self, string):
-        """render template from string"""
+        """
+        render template from string
+        may raise a UndefinedException
+        in case a variable is undefined
+        """
         if not string:
             return ''
-        return self.env.from_string(string).render()
+        try:
+            return self.env.from_string(string).render(self.variables)
+        except UndefinedError as e:
+            err = 'undefined variable: {}'.format(e.message)
+            raise UndefinedException(err)
 
     def add_tmp_vars(self, newvars={}):
         """add vars to the globals, make sure to call restore_vars"""
-        saved_globals = self.env.globals.copy()
+        saved_variables = self.variables.copy()
         if not newvars:
-            return saved_globals
-        self.env.globals.update(newvars)
-        return saved_globals
+            return saved_variables
+        self.variables.update(newvars)
+        return saved_variables
 
     def restore_vars(self, saved_globals):
         """restore globals from add_tmp_vars"""
-        self.env.globals = saved_globals.copy()
+        self.variables = saved_globals.copy()
 
     def update_variables(self, variables):
         """update variables"""
-        self.env.globals.update(variables)
+        self.variables.update(variables)
 
     def _load_path_to_dic(self, path, dic):
         mod = utils.get_module_from_path(path)
@@ -160,7 +184,7 @@ class Templategen:
         template_rel_path = os.path.relpath(src, self.base)
         try:
             template = self.env.get_template(template_rel_path)
-            content = template.render()
+            content = template.render(self.variables)
         except UnicodeDecodeError:
             data = self._read_bad_encoded_text(src)
             content = self.generate_string(data)

dotdrop/updater.py

@@ -14,6 +14,7 @@ from dotdrop.logger import Logger
 from dotdrop.templategen import Templategen
 from dotdrop.utils import patch_ignores, remove, get_unique_tmp_name, \
     write_to_tmpfile, must_ignore, mirror_file_rights
+from dotdrop.exceptions import UndefinedException
 
 
 TILD = '~'
@@ -186,7 +187,11 @@ class Updater:
             if self.debug:
                 self.log.dbg('{} is a template'.format(dtpath))
             if self.showpatch:
-                self._show_patch(path, dtpath)
+                try:
+                    self._show_patch(path, dtpath)
+                except UndefinedException as e:
+                    msg = 'unable to show patch for {}: {}'.format(path, e)
+                    self.log.warn(msg)
             return False
         if compare and filecmp.cmp(path, dtpath, shallow=False) and \
                 self._same_rights(path, dtpath):

tests-ng/actions-args-template.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/actions-args.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/actions-default.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/actions-pre.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/actions-template.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/actions.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/compare-ignore-relative.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"
@@ -66,9 +67,9 @@ create_conf ${cfg} # sets token
 
 # import
 echo "[+] import"
-cd ${ddpath} | ${bin} import -c ${cfg} ${tmpd}/program
-cd ${ddpath} | ${bin} import -c ${cfg} ${tmpd}/config
-cd ${ddpath} | ${bin} import -c ${cfg} ${tmpd}/vscode
+cd ${ddpath} | ${bin} import --verbose -c ${cfg} ${tmpd}/program || exit 1
+cd ${ddpath} | ${bin} import --verbose -c ${cfg} ${tmpd}/config || exit 1
+cd ${ddpath} | ${bin} import --verbose -c ${cfg} ${tmpd}/vscode || exit 1
 
 # add files on filesystem
 echo "[+] add files"

tests-ng/compare-ignore.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/compare.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/corner-case.sh

@@ -0,0 +1,102 @@
+#!/usr/bin/env bash
+# author: deadc0de6 (https://github.com/deadc0de6)
+# Copyright (c) 2019, deadc0de6
+#
+# the only purpose is to test corner-cases
+# not covered by other tests like
+# dry
+# diff before write
+# etc
+#
+# returns 1 in case of error
+#
+
+# exit on first error
+#set -e
+
+# all this crap to get current path
+rl="readlink -f"
+if ! ${rl} "${0}" >/dev/null 2>&1; then
+  rl="realpath"
+
+  if ! hash ${rl}; then
+    echo "\"${rl}\" not found !" && exit 1
+  fi
+fi
+cur=$(dirname "$(${rl} "${0}")")
+
+#hash dotdrop >/dev/null 2>&1
+#[ "$?" != "0" ] && echo "install dotdrop to run tests" && exit 1
+
+#echo "called with ${1}"
+
+# dotdrop path can be pass as argument
+ddpath="${cur}/../"
+[ "${1}" != "" ] && ddpath="${1}"
+[ ! -d ${ddpath} ] && echo "ddpath \"${ddpath}\" is not a directory" && exit 1
+
+export PYTHONPATH="${ddpath}:${PYTHONPATH}"
+bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
+
+echo "dotdrop path: ${ddpath}"
+echo "pythonpath: ${PYTHONPATH}"
+
+# get the helpers
+source ${cur}/helpers
+
+echo -e "$(tput setaf 6)==> RUNNING $(basename $BASH_SOURCE) <==$(tput sgr0)"
+
+################################################################
+# this is the test
+################################################################
+
+# dotdrop directory
+basedir=`mktemp -d --suffix='-dotdrop-tests' || mktemp -d`
+echo "[+] dotdrop dir: ${basedir}"
+echo "[+] dotpath dir: ${basedir}/dotfiles"
+
+# create the config file
+cfg="${basedir}/config.yaml"
+cat > ${cfg} << _EOF
+config:
+  backup: true
+  create: true
+  dotpath: dotfiles
+dotfiles:
+  f_x:
+    src: /tmp/x
+    dst:
+  f_y:
+    src: /tmp/.i-do-not-exist-dotdrop
+    dst: /tmp/y
+profiles:
+  p1:
+    dotfiles:
+    - f_x
+    - f_y
+
+_EOF
+
+echo "[+] test install dry"
+cd ${ddpath} | ${bin} install -c ${cfg} --dry -p p1 --verbose f_x
+[ "$?" != "0" ] && exit 1
+
+echo "[+] test install show-diff"
+cd ${ddpath} | ${bin} install -c ${cfg} -p p1 --verbose f_x
+[ "$?" != "0" ] && exit 1
+cd ${ddpath} | ${bin} install -D -c ${cfg} -p p1 --verbose f_x
+[ "$?" != "0" ] && exit 1
+
+echo "[+] test install not existing src"
+cd ${ddpath} | ${bin} install -c ${cfg} --dry -p p1 --verbose f_y
+
+echo "[+] test install to temp"
+cd ${ddpath} | ${bin} install -t -c ${cfg} -p p1 --verbose f_x
+[ "$?" != "0" ] && exit 1
+
+## CLEANING
+rm -rf ${basedir}
+
+echo "OK"
+exit 0

tests-ng/deprecated-link.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/diff-cmd.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"
@@ -67,6 +68,9 @@ dotfiles:
 profiles:
 _EOF
 
+export DOTDROP_FORCE_NODEBUG=yes
+export DOTDROP_NOBANNER=yes
+
 # import
 echo "[+] import"
 cd ${ddpath} | ${bin} import -c ${cfg} ${tmpd}/singlefile
@@ -74,9 +78,6 @@ cd ${ddpath} | ${bin} import -c ${cfg} ${tmpd}/singlefile
 # modify the file
 echo "modified" > ${tmpd}/singlefile
 
-# suppressing the banner, so we can compare dotdrop diff with UNIX diff
-export DOTDROP_NOBANNER=yes
-
 # default diff (unified)
 echo "[+] comparing with default diff (unified)"
 set +e
@@ -121,7 +122,9 @@ grep fakediff ${tmpd}/fake &> /dev/null || exit 1
 
 ## CLEANING
 rm -rf ${basedir} ${tmpd}
+
 unset DOTDROP_NOBANNER
+unset DOTDROP_FORCE_NODEBUG
 
 echo "OK"
 exit 0

tests-ng/dir-import-update.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/dotdrop-variables.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/dotfile-no-src.sh

@@ -33,6 +33,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/dotfile-sub-variables.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/dotfile-variables.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/dotfiles-all.sh

@@ -33,6 +33,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/dotfiles-dyn-paths.sh

@@ -33,6 +33,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/duplicate-key.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"
@@ -86,7 +87,7 @@ mkdir -p ${tmpd}/sub/sub
 echo "test2" > ${tmpd}/sub/sub/abc
 
 # import
-cd ${ddpath} | ${bin} import -c ${cfg} -p p2 \
+cd ${ddpath} | ${bin} import --verbose -c ${cfg} -p p2 \
   ${tmpd}/abc \
   ${tmpd}/sub/abc \
   ${tmpd}/sub/abc \
@@ -94,7 +95,7 @@ cd ${ddpath} | ${bin} import -c ${cfg} -p p2 \
   ${tmpd}/sub/sub2/abc
 
 # count dotfiles for p2
-cnt=`cd ${ddpath} | ${bin} files -c ${cfg} -p p2 -b | grep '^f_' | wc -l`
+cnt=`cd ${ddpath} | ${bin} files --verbose -c ${cfg} -p p2 -b | grep '^f_' | wc -l`
 [ "${cnt}" != "4" ] && exit 1
 
 ## CLEANING

tests-ng/dynactions.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/dyndotfilepaths.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/dynextvariables.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/dyninclude.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/dynvariables.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"
@@ -68,10 +69,12 @@ config:
   dotpath: dotfiles
 variables:
   var1: "this is some test"
+  var2: "the_dvar4"
 dynvariables:
   dvar1: head -1 /proc/meminfo
   dvar2: "echo 'this is some test' | rev | tr ' ' ','"
   dvar3: ${scr}
+  dvar4: "echo {{@@ var2 @@}} | rev"
 dotfiles:
   f_abc:
     dst: ${tmpd}/abc
@@ -88,17 +91,19 @@ echo "{{@@ var1 @@}}" > ${tmps}/dotfiles/abc
 echo "{{@@ dvar1 @@}}" >> ${tmps}/dotfiles/abc
 echo "{{@@ dvar2 @@}}" >> ${tmps}/dotfiles/abc
 echo "{{@@ dvar3 @@}}" >> ${tmps}/dotfiles/abc
+echo "{{@@ dvar4 @@}}" >> ${tmps}/dotfiles/abc
 echo "test" >> ${tmps}/dotfiles/abc
 
 # install
 cd ${ddpath} | ${bin} install -f -c ${cfg} -p p1 -V
 
-#cat ${tmpd}/abc
+cat ${tmpd}/abc
 
 grep '^this is some test' ${tmpd}/abc >/dev/null
 grep "^MemTotal" ${tmpd}/abc >/dev/null
 grep '^tset,emos,si,siht' ${tmpd}/abc >/dev/null
 grep "^${TESTENV}" ${tmpd}/abc > /dev/null
+grep '^4ravd_eht' ${tmpd}/abc >/dev/null
 
 #cat ${tmpd}/abc
 

tests-ng/ext-actions.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/extvariables.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/filter_file.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/force-actions.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/func_file.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/global-compare-ignore.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/global-update-ignore.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/globs.sh

@@ -35,6 +35,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/header.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/ignore-empty.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"
@@ -76,7 +77,7 @@ _EOF
 
 # create the dotfile
 mkdir -p ${tmps}/dotfiles/d1
-echo "{{@@ var1 @@}}" > ${tmps}/dotfiles/d1/empty
+echo "{#@@ should be stripped @@#}" > ${tmps}/dotfiles/d1/empty
 echo "not empty" > ${tmps}/dotfiles/d1/notempty
 
 # install

tests-ng/import-as.sh

@@ -31,6 +31,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/import-configs.sh

@@ -31,6 +31,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/import-duplicate.sh

@@ -31,6 +31,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/import-link-children.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/import-non-existing.sh

@@ -31,6 +31,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/import-profile-dotfiles.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/import-subfile.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/import-with-empty.sh

@@ -0,0 +1,113 @@
+#!/usr/bin/env bash
+# author: deadc0de6 (https://github.com/deadc0de6)
+# Copyright (c) 2019, deadc0de6
+#
+# test import new dotfiles with empty dst/src on existing dotfiles
+# returns 1 in case of error
+#
+
+# exit on first error
+set -e
+
+# all this crap to get current path
+rl="readlink -f"
+if ! ${rl} "${0}" >/dev/null 2>&1; then
+  rl="realpath"
+
+  if ! hash ${rl}; then
+    echo "\"${rl}\" not found !" && exit 1
+  fi
+fi
+cur=$(dirname "$(${rl} "${0}")")
+
+#hash dotdrop >/dev/null 2>&1
+#[ "$?" != "0" ] && echo "install dotdrop to run tests" && exit 1
+
+#echo "called with ${1}"
+
+# dotdrop path can be pass as argument
+ddpath="${cur}/../"
+[ "${1}" != "" ] && ddpath="${1}"
+[ ! -d ${ddpath} ] && echo "ddpath \"${ddpath}\" is not a directory" && exit 1
+
+export PYTHONPATH="${ddpath}:${PYTHONPATH}"
+bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
+
+echo "dotdrop path: ${ddpath}"
+echo "pythonpath: ${PYTHONPATH}"
+
+# get the helpers
+source ${cur}/helpers
+
+echo -e "$(tput setaf 6)==> RUNNING $(basename $BASH_SOURCE) <==$(tput sgr0)"
+
+################################################################
+# this is the test
+################################################################
+
+# dotdrop directory
+basedir=`mktemp -d --suffix='-dotdrop-tests' || mktemp -d`
+echo "[+] dotdrop dir: ${basedir}"
+echo "[+] dotpath dir: ${basedir}/dotfiles"
+# the temp directory
+tmpd=`mktemp -d --suffix='-dotdrop-tests' || mktemp -d`
+
+# create a dotfile
+dftoimport="${tmpd}/a_dotfile"
+echo 'some content' > ${dftoimport}
+
+# create the config file
+cfg="${basedir}/config.yaml"
+cat > ${cfg} << _EOF
+config:
+  backup: true
+  create: true
+  dotpath: dotfiles
+dotfiles:
+  f_x:
+    src: /tmp/x
+    dst:
+  f_y:
+    src:
+    dst: /tmp/y
+  f_z:
+    src:
+    dst:
+  f_l:
+    src:
+    dst:
+    link: link
+  f_lc:
+    src:
+    dst:
+    link: link_children
+profiles:
+  p1:
+    dotfiles:
+    - f_x
+    - f_y
+    - f_z
+    - f_l
+    - f_lc
+_EOF
+
+echo "[+] import"
+cd ${ddpath} | ${bin} import -c ${cfg} -p p1 --verbose ${dftoimport}
+[ "$?" != "0" ] && exit 1
+
+echo "[+] install"
+cd ${ddpath} | ${bin} install -c ${cfg} -p p1 --verbose | grep '^5 dotfile(s) installed.$'
+rm -f ${dftoimport}
+cd ${ddpath} | ${bin} install -c ${cfg} -p p1 --verbose | grep '^6 dotfile(s) installed.$'
+
+nb=`cd ${ddpath} | ${bin} files -c ${cfg} -p p1 --verbose | grep '^[a-zA-Z]' | wc -l`
+[ "${nb}" != "6" ] && echo 'error in dotfile list' && exit 1
+
+#cat ${cfg}
+
+## CLEANING
+rm -rf ${basedir} ${tmpd}
+
+echo "OK"
+exit 0

tests-ng/import.sh

@@ -31,6 +31,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/imported-configs-variables.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/imported-variables-from-config.sh

@@ -0,0 +1,110 @@
+#!/usr/bin/env bash
+# author: davla (https://github.com/davls)
+# Copyright (c) 2020, davla
+#
+# test variables imported from config and used in the importing yaml config
+# returns 1 in case of error
+#
+
+# exit on first error
+set -e
+
+# all this crap to get current path
+rl="readlink -f"
+if ! ${rl} "${0}" >/dev/null 2>&1; then
+  rl="realpath"
+
+  if ! hash ${rl}; then
+    echo "\"${rl}\" not found !" && exit 1
+  fi
+fi
+cur=$(dirname "$(${rl} "${0}")")
+
+#hash dotdrop >/dev/null 2>&1
+#[ "$?" != "0" ] && echo "install dotdrop to run tests" && exit 1
+
+#echo "called with ${1}"
+
+# dotdrop path can be pass as argument
+ddpath="${cur}/../"
+[ "${1}" != "" ] && ddpath="${1}"
+[ ! -d ${ddpath} ] && echo "ddpath \"${ddpath}\" is not a directory" && exit 1
+
+export PYTHONPATH="${ddpath}:${PYTHONPATH}"
+bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
+
+echo "dotdrop path: ${ddpath}"
+echo "pythonpath: ${PYTHONPATH}"
+
+# get the helpers
+source ${cur}/helpers
+
+echo -e "$(tput setaf 6)==> RUNNING $(basename $BASH_SOURCE) <==$(tput sgr0)"
+
+################################################################
+# this is the test
+################################################################
+
+# the dotfile source
+tmps=`mktemp -d --suffix='-dotdrop-tests' || mktemp -d`
+mkdir -p ${tmps}/dotfiles
+# the dotfile destination
+tmpd=`mktemp -d --suffix='-dotdrop-tests' || mktemp -d`
+
+# create the config file
+cfg="${tmps}/config.yaml"
+subcfg="${tmps}/subconfig.yaml"
+
+cat > ${cfg} << _EOF
+config:
+  backup: true
+  create: true
+  dotpath: dotfiles
+  import_configs:
+  - ${subcfg}
+dotfiles:
+  f_abc:
+    dst: ${tmpd}/abc
+    src: '{{@@ abc_dyn_src @@}}{{@@ abc_src @@}}'
+profiles:
+  p1:
+    dotfiles:
+    - f_abc
+_EOF
+cat ${cfg}
+
+# create the subconfig file
+cat > ${subcfg} << _EOF
+config:
+  backup: true
+  create: true
+  dotpath: dotfiles
+variables:
+  abc_src: c
+dynvariables:
+  abc_dyn_src: 'echo ab'
+dotfiles: []
+profiles: []
+_EOF
+
+# create the dotfile
+dirname ${tmps}/dotfiles/abc | xargs mkdir -p
+cat > ${tmps}/dotfiles/abc << _EOF
+Hell yeah
+_EOF
+
+# install
+cd ${ddpath} | ${bin} install -f -c ${cfg} -p p1 -V
+
+# test file existence and content
+[ -f "${tmpd}/abc" ] || {
+    echo 'Dotfile not installed'
+    exit 1
+}
+
+## CLEANING
+rm -rf ${tmps} ${tmpd}
+
+echo "OK"
+exit 0

tests-ng/include-actions.sh

@@ -33,6 +33,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/include-order.sh

@@ -33,6 +33,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/include.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"
@@ -108,6 +109,9 @@ cd ${ddpath} | ${bin} files -c ${cfg} -p p2 | grep f_abc
 cd ${ddpath} | ${bin} files -c ${cfg} -p p3 | grep f_abc
 cd ${ddpath} | ${bin} files -c ${cfg} -p p0 | grep f_abc
 
+cnt=`cd ${ddpath} | ${bin} files -c ${cfg} -p p0 | grep f_abc | wc -l`
+[ "${cnt}" != "1" ] && echo "dotfiles displayed more than once" && exit 1
+
 # count
 cnt=`cd ${ddpath} | ${bin} files -c ${cfg} -p p1 -b | grep '^f_' | wc -l`
 [ "${cnt}" != "1" ] && exit 1

tests-ng/inst-link-default.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/install-empty.sh

@@ -7,7 +7,7 @@
 #
 
 # exit on first error
-#set -e
+set -e
 
 # all this crap to get current path
 rl="readlink -f"
@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"
@@ -67,17 +68,26 @@ dotfiles:
   f_z:
     src:
     dst:
+  f_l:
+    src:
+    dst:
+    link: link
+  f_lc:
+    src:
+    dst:
+    link: link_children
 profiles:
   p1:
     dotfiles:
     - f_x
     - f_y
     - f_z
-
+    - f_l
+    - f_lc
 _EOF
 
 echo "[+] install"
-cd ${ddpath} | ${bin} install -c ${cfg} -p p1 --verbose | grep '^3 dotfile(s) installed.$'
+cd ${ddpath} | ${bin} install -c ${cfg} -p p1 --verbose | grep '^5 dotfile(s) installed.$'
 [ "$?" != "0" ] && exit 1
 
 ## CLEANING

tests-ng/install-ignore.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"
@@ -81,7 +82,7 @@ echo "new data" > ${basedir}/dotfiles/${tmpd}/readmes/README.md
 # install
 rm -rf ${tmpd}
 echo "[+] install normal"
-cd ${ddpath} | ${bin} install -c ${cfg} --verbose
+cd ${ddpath} | ${bin} install --showdiff -c ${cfg} --verbose
 [ "$?" != "0" ] && exit 1
 nb=`find ${tmpd} -iname 'README.md' | wc -l`
 echo "(1) found ${nb} README.md file(s)"

tests-ng/install-to-temp.sh

@@ -0,0 +1,89 @@
+#!/usr/bin/env bash
+# author: deadc0de6 (https://github.com/deadc0de6)
+# Copyright (c) 2019, deadc0de6
+#
+# test install to temp
+# returns 1 in case of error
+#
+
+# exit on first error
+set -e
+
+# all this crap to get current path
+rl="readlink -f"
+if ! ${rl} "${0}" >/dev/null 2>&1; then
+  rl="realpath"
+
+  if ! hash ${rl}; then
+    echo "\"${rl}\" not found !" && exit 1
+  fi
+fi
+cur=$(dirname "$(${rl} "${0}")")
+
+#hash dotdrop >/dev/null 2>&1
+#[ "$?" != "0" ] && echo "install dotdrop to run tests" && exit 1
+
+#echo "called with ${1}"
+
+# dotdrop path can be pass as argument
+ddpath="${cur}/../"
+[ "${1}" != "" ] && ddpath="${1}"
+[ ! -d ${ddpath} ] && echo "ddpath \"${ddpath}\" is not a directory" && exit 1
+
+export PYTHONPATH="${ddpath}:${PYTHONPATH}"
+bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
+
+echo "dotdrop path: ${ddpath}"
+echo "pythonpath: ${PYTHONPATH}"
+
+# get the helpers
+source ${cur}/helpers
+
+echo -e "$(tput setaf 6)==> RUNNING $(basename $BASH_SOURCE) <==$(tput sgr0)"
+
+################################################################
+# this is the test
+################################################################
+
+# dotdrop directory
+basedir=`mktemp -d --suffix='-dotdrop-tests' || mktemp -d`
+mkdir -p ${basedir}/dotfiles
+tmpd=`mktemp -d --suffix='-dotdrop-tests' || mktemp -d`
+echo "[+] dotdrop dir: ${basedir}"
+echo "[+] dotpath dir: ${basedir}/dotfiles"
+
+# create the config file
+cfg="${basedir}/config.yaml"
+cat > ${cfg} << _EOF
+config:
+  backup: true
+  create: true
+  dotpath: dotfiles
+dotfiles:
+  f_x:
+    src: x
+    dst: ${tmpd}/x
+  f_y:
+    src: y
+    dst: ${tmpd}/y
+    link: link
+profiles:
+  p1:
+    dotfiles:
+    - f_x
+    - f_y
+_EOF
+
+echo 'test_x' > ${basedir}/dotfiles/x
+echo 'test_y' > ${basedir}/dotfiles/y
+
+echo "[+] install"
+cd ${ddpath} | ${bin} install -c ${cfg} -p p1 --showdiff --verbose --temp | grep '^2 dotfile(s) installed.$'
+[ "$?" != "0" ] && exit 1
+
+## CLEANING
+rm -rf ${basedir}
+
+echo "OK"
+exit 0

tests-ng/jhelpers.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/link-import-default.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/link-templates-dir-home.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/link-templates-dir.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/link-templates.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/link-value-tests.sh

@@ -33,6 +33,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/macro-with-globals.sh

@@ -0,0 +1,102 @@
+#!/usr/bin/env bash
+# author: deadc0de6 (https://github.com/deadc0de6)
+# Copyright (c) 2019, deadc0de6
+#
+# import variables from file
+#
+
+# exit on first error
+set -e
+
+# all this crap to get current path
+rl="readlink -f"
+if ! ${rl} "${0}" >/dev/null 2>&1; then
+  rl="realpath"
+
+  if ! hash ${rl}; then
+    echo "\"${rl}\" not found !" && exit 1
+  fi
+fi
+cur=$(dirname "$(${rl} "${0}")")
+
+#hash dotdrop >/dev/null 2>&1
+#[ "$?" != "0" ] && echo "install dotdrop to run tests" && exit 1
+
+#echo "called with ${1}"
+
+# dotdrop path can be pass as argument
+ddpath="${cur}/../"
+[ "${1}" != "" ] && ddpath="${1}"
+[ ! -d ${ddpath} ] && echo "ddpath \"${ddpath}\" is not a directory" && exit 1
+
+export PYTHONPATH="${ddpath}:${PYTHONPATH}"
+bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
+
+echo "dotdrop path: ${ddpath}"
+echo "pythonpath: ${PYTHONPATH}"
+
+# get the helpers
+source ${cur}/helpers
+
+echo -e "$(tput setaf 6)==> RUNNING $(basename $BASH_SOURCE) <==$(tput sgr0)"
+
+################################################################
+# this is the test
+################################################################
+
+# the dotfile source
+tmps=`mktemp -d --suffix='-dotdrop-tests' || mktemp -d`
+mkdir -p ${tmps}/dotfiles
+# the dotfile destination
+tmpd=`mktemp -d --suffix='-dotdrop-tests' || mktemp -d`
+
+# create the config file
+cfg="${tmps}/config.yaml"
+
+cat > ${cfg} << _EOF
+config:
+  backup: true
+  create: true
+  dotpath: dotfiles
+dotfiles:
+  f_abc:
+    dst: ${tmpd}/abc
+    src: abc
+profiles:
+  p0:
+    dotfiles:
+    - f_abc
+variables:
+  global: global_var
+  local: local_var
+_EOF
+
+# create the source
+mkdir -p ${tmps}/dotfiles/
+
+cat > ${tmps}/dotfiles/macro_file << _EOF
+{%@@ macro macro(var) @@%}
+{{@@ global @@}}
+{{@@ var @@}}
+{%@@ endmacro @@%}
+_EOF
+
+cat > ${tmps}/dotfiles/abc << _EOF
+{%@@ from 'macro_file' import macro with context @@%}
+{{@@ macro(local) @@}}
+_EOF
+
+# install
+cd ${ddpath} | ${bin} install -c ${cfg} -p p0 -V
+
+# test file content
+cat ${tmpd}/abc
+grep 'global_var' ${tmpd}/abc >/dev/null 2>&1
+grep 'local_var' ${tmpd}/abc >/dev/null 2>&1
+
+## CLEANING
+rm -rf ${tmps} ${tmpd}
+
+echo "OK"
+exit 0

tests-ng/minversion.sh

@@ -30,6 +30,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/profile-actions.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/profile-dyninclude.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"
@@ -90,6 +91,9 @@ dotfiles:
   f_def:
     dst: ${tmpd}/def
     src: def
+  f_ghi:
+    dst: '${tmpd}/{{@@ ghi @@}}'
+    src: ghi
 variables:
   mainvar: 'bad0'
   subvar: 'bad1'
@@ -100,8 +104,10 @@ profiles:
   subprofile:
     dotfiles:
     - f_abc
+    - f_ghi
     dynvariables:
       subdyn: 'echo subdyncontent'
+      ghi: 'echo ghi'
     variables:
       subvar: 'subcontent'
   subignore:
@@ -118,6 +124,7 @@ echo "{{@@ subdyn @@}}" >> ${tmps}/dotfiles/abc
 echo "{{@@ subvar @@}}" >> ${tmps}/dotfiles/abc
 echo "end" >> ${tmps}/dotfiles/abc
 #cat ${tmps}/dotfiles/abc
+echo "ghi content" > ${tmps}/dotfiles/ghi
 
 # install
 cd ${ddpath} | ${bin} install -f -c ${cfg} -p profile_1 --verbose
@@ -129,6 +136,7 @@ grep 'maindyncontent' ${tmpd}/abc >/dev/null || (echo "dynvariables 1 not resolv
 grep 'subcontent' ${tmpd}/abc >/dev/null || (echo "variables 2 not resolved" && exit 1)
 grep 'subdyncontent' ${tmpd}/abc >/dev/null || (echo "dynvariables 2 not resolved" && exit 1)
 #cat ${tmpd}/abc
+[ ! -e ${tmpd}/ghi ] && exit 1
 
 ## CLEANING
 rm -rf ${tmps} ${tmpd}

tests-ng/profile-dynvariables.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/profile-undefined-variables.sh

@@ -0,0 +1,124 @@
+#!/usr/bin/env bash
+# author: deadc0de6 (https://github.com/deadc0de6)
+# Copyright (c) 2017, deadc0de6
+#
+# test variables defined in a different profile
+# than the one selected
+# returns 1 in case of error
+#
+
+# exit on first error
+set -e
+
+# all this crap to get current path
+rl="readlink -f"
+if ! ${rl} "${0}" >/dev/null 2>&1; then
+  rl="realpath"
+
+  if ! hash ${rl}; then
+    echo "\"${rl}\" not found !" && exit 1
+  fi
+fi
+cur=$(dirname "$(${rl} "${0}")")
+
+#hash dotdrop >/dev/null 2>&1
+#[ "$?" != "0" ] && echo "install dotdrop to run tests" && exit 1
+
+#echo "called with ${1}"
+
+# dotdrop path can be pass as argument
+ddpath="${cur}/../"
+[ "${1}" != "" ] && ddpath="${1}"
+[ ! -d ${ddpath} ] && echo "ddpath \"${ddpath}\" is not a directory" && exit 1
+
+export PYTHONPATH="${ddpath}:${PYTHONPATH}"
+bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
+
+echo "dotdrop path: ${ddpath}"
+echo "pythonpath: ${PYTHONPATH}"
+
+# get the helpers
+source ${cur}/helpers
+
+echo -e "$(tput setaf 6)==> RUNNING $(basename $BASH_SOURCE) <==$(tput sgr0)"
+
+################################################################
+# this is the test
+################################################################
+
+# the dotfile source
+tmps=`mktemp -d --suffix='-dotdrop-tests' || mktemp -d`
+mkdir -p ${tmps}/dotfiles
+# the dotfile destination
+tmpd=`mktemp -d --suffix='-dotdrop-tests' || mktemp -d`
+#echo "dotfile destination: ${tmpd}"
+
+# create the config file
+cfg="${tmps}/config.yaml"
+
+cat > ${cfg} << _EOF
+config:
+  backup: true
+  create: true
+  dotpath: dotfiles
+dotfiles:
+  f_abc:
+    dst: "${tmpd}/{{@@ defined_in_main @@}}"
+    src: abc
+  f_def:
+    dst: "${tmpd}/{{@@ defined_in_alt @@}}"
+    src: def
+profiles:
+  pmain:
+    dynvariables:
+      defined_in_main: echo abc
+    dotfiles:
+    - f_abc
+  palt:
+    dynvariables:
+      defined_in_alt: echo def
+    dotfiles:
+    - f_def
+  pall:
+    dynvariables:
+      defined_in_main: echo abcall
+      defined_in_alt: echo defall
+    dotfiles:
+    - ALL
+  pinclude:
+    include:
+    - pmain
+_EOF
+#cat ${cfg}
+
+# create the dotfile
+echo "main" > ${tmps}/dotfiles/abc
+echo "alt" > ${tmps}/dotfiles/def
+
+# install pmain
+echo "install pmain"
+cd ${ddpath} | ${bin} install -f -c ${cfg} -p pmain -V
+[ ! -e ${tmpd}/abc ] && echo "dotfile not installed" && exit 1
+grep main ${tmpd}/abc
+
+# install pall
+echo "install pall"
+cd ${ddpath} | ${bin} install -f -c ${cfg} -p pall -V
+[ ! -e ${tmpd}/abcall ] && echo "dotfile not installed" && exit 1
+grep main ${tmpd}/abcall
+[ ! -e ${tmpd}/defall ] && echo "dotfile not installed" && exit 1
+grep alt ${tmpd}/defall
+
+# install pinclude
+echo "install pinclude"
+rm -f ${tmpd}/abc
+cd ${ddpath} | ${bin} install -f -c ${cfg} -p pinclude -V
+[ ! -e ${tmpd}/abc ] && echo "dotfile not installed" && exit 1
+grep main ${tmpd}/abc
+
+## CLEANING
+rm -rf ${tmps} ${tmpd} ${scr} ${scr2}
+
+echo "OK"
+exit 0

tests-ng/re-import.sh

@@ -31,6 +31,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/recinclude.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/recvariables.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/remove.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/symlink.sh

@@ -31,6 +31,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/transformations-template.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/transformations-with-args.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/transformations.sh

@@ -34,6 +34,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/update-ignore-relative.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/update-ignore.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/update-rights.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/update-templates.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/update-with-key.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/update.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/variables-include.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests-ng/variables.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"
@@ -55,6 +56,7 @@ tmpd=`mktemp -d --suffix='-dotdrop-tests' || mktemp -d`
 
 # create the config file
 cfg="${tmps}/config.yaml"
+export dotdrop_test_dst="${tmpd}/def"
 
 cat > ${cfg} << _EOF
 config:
@@ -65,14 +67,19 @@ variables:
   var1: "this is some test"
   var2: 12
   var3: another test
+  vardst: "{{@@ env['dotdrop_test_dst'] @@}}"
 dotfiles:
   f_abc:
     dst: ${tmpd}/abc
     src: abc
+  f_def:
+    dst: "{{@@ vardst @@}}"
+    src: def
 profiles:
   p1:
     dotfiles:
     - f_abc
+    - f_def
 _EOF
 #cat ${cfg}
 
@@ -82,13 +89,19 @@ echo "{{@@ var2 @@}}" >> ${tmps}/dotfiles/abc
 echo "{{@@ var3 @@}}" >> ${tmps}/dotfiles/abc
 echo "test" >> ${tmps}/dotfiles/abc
 
-# install
-cd ${ddpath} | ${bin} install -f -c ${cfg} -p p1
+echo "test_def" > ${tmps}/dotfiles/def
 
+# install
+cd ${ddpath} | ${bin} install -f -c ${cfg} -p p1 --verbose
+
+[ ! -e ${tmpd}/abc ] && echo "abc not installed" && exit 1
 grep '^this is some test' ${tmpd}/abc >/dev/null
 grep '^12' ${tmpd}/abc >/dev/null
 grep '^another test' ${tmpd}/abc >/dev/null
 
+[ ! -e ${tmpd}/def ] && echo "def not installed" && exit 1
+grep '^test_def' ${tmpd}/def >/dev/null
+
 #cat ${tmpd}/abc
 
 ## CLEANING

tests-ng/workdir.sh

@@ -32,6 +32,7 @@ ddpath="${cur}/../"
 
 export PYTHONPATH="${ddpath}:${PYTHONPATH}"
 bin="python3 -m dotdrop.dotdrop"
+hash coverage 2>/dev/null && bin="coverage run -a --source=dotdrop -m dotdrop.dotdrop" || true
 
 echo "dotdrop path: ${ddpath}"
 echo "pythonpath: ${PYTHONPATH}"

tests.sh

@@ -6,37 +6,52 @@
 set -ev
 
 # PEP8 tests
-which pycodestyle 2>/dev/null
+which pycodestyle >/dev/null 2>&1
 [ "$?" != "0" ] && echo "Install pycodestyle" && exit 1
+echo "testing with pycodestyle"
 pycodestyle --ignore=W503,W504,W605 dotdrop/
 pycodestyle tests/
 pycodestyle scripts/
 
 # pyflakes tests
+echo "testing with pyflakes"
 pyflakes dotdrop/
 pyflakes tests/
 
 # retrieve the nosetests binary
-set +e
 nosebin="nosetests"
-which ${nosebin} 2>/dev/null
+which ${nosebin} >/dev/null 2>&1
 [ "$?" != "0" ] && nosebin="nosetests3"
-which ${nosebin} 2>/dev/null
+which ${nosebin} >/dev/null 2>&1
 [ "$?" != "0" ] && echo "Install nosetests" && exit 1
-set -e
 
 # do not print debugs when running tests (faster)
 export DOTDROP_FORCE_NODEBUG=yes
 
+# coverage file location
+cur=`dirname $(readlink -f "${0}")`
+export COVERAGE_FILE="${cur}/.coverage"
+
 # execute tests with coverage
-PYTHONPATH=dotdrop ${nosebin} -s --with-coverage --cover-package=dotdrop
-#PYTHONPATH=dotdrop python3 -m pytest tests
+PYTHONPATH="dotdrop" ${nosebin} -s --with-coverage --cover-package=dotdrop
+#PYTHONPATH="dotdrop" python3 -m pytest tests
+
+# enable debug logs
+export DOTDROP_DEBUG=
+unset DOTDROP_FORCE_NODEBUG
+# do not print debugs when running tests (faster)
+#export DOTDROP_FORCE_NODEBUG=yes
 
 ## execute bash script tests
 [ "$1" = '--python-only' ] || {
+  echo "doing extended tests"
   log=`mktemp`
   for scr in tests-ng/*.sh; do
-    ${scr} > "${log}" 2>&1 &
+    if [ -z ${TRAVIS} ]; then
+      ${scr} > "${log}" 2>&1 &
+    else
+      ${scr} > "${log}" >/dev/null 2>&1 &
+    fi
     tail --pid="$!" -f "${log}"
     set +e
     wait "$!"

tests/dummy.py

@@ -0,0 +1,22 @@
+"""
+author: deadc0de6 (https://github.com/deadc0de6)
+Copyright (c) 2017, deadc0de6
+basic unittest for the import function
+"""
+
+
+import unittest
+import dotdrop
+
+
+class TestDummy(unittest.TestCase):
+
+    dotdrop.main()
+
+
+def main():
+    unittest.main()
+
+
+if __name__ == '__main__':
+    main()

tests/helpers.py

@@ -158,7 +158,6 @@ def load_options(confpath, profile):
     o.import_link = LinkTypes.NOLINK
     o.install_showdiff = True
     o.debug = True
-    o.variables = {}
     return o
 
 

tests/test_compare.py

@@ -36,8 +36,8 @@ class TestCompare(unittest.TestCase):
         results = {}
         for dotfile in dotfiles:
             path = os.path.expanduser(dotfile.dst)
-            ret, insttmp = inst.install_to_temp(t, tmp, dotfile.src,
-                                                dotfile.dst)
+            ret, err, insttmp = inst.install_to_temp(t, tmp, dotfile.src,
+                                                     dotfile.dst)
             if not ret:
                 results[path] = False
                 continue

tests/test_install.py

@@ -186,7 +186,6 @@ exec bspwm
         o = load_options(confpath, profile)
         o.safe = False
         o.install_showdiff = True
-        o.variables = {}
         cmd_install(o)
 
         # now compare the generated files
@@ -363,8 +362,8 @@ exec bspwm
         src = '/some/non/existant/file'
 
         installer = Installer()
-        logger = MagicMock()
-        installer.log.err = logger
+        # logger = MagicMock()
+        # installer.log.err = logger
 
         res, err = installer.link_children(templater=MagicMock(), src=src,
                                            dst='/dev/null', actionexec=None)
@@ -382,10 +381,10 @@ exec bspwm
 
         src = create_random_file(src_dir)[0]
 
-        logger = MagicMock()
+        # logger = MagicMock()
         templater = MagicMock()
         installer = Installer()
-        installer.log.err = logger
+        # installer.log.err = logger
 
         # pass src file not src dir
         res, err = installer.link_children(templater=templater, src=src,