mirrors/dotdrop
1
0
Fork 0
mirror of https://github.com/deadc0de6/dotdrop.git synced 2026-02-04 11:01:45 +00:00
Code Activity

deprecated trans_r/w for install/update

Browse Source
This commit is contained in:
deadc0de6
2023-09-22 17:37:51 +02:00
committed by deadc0de
parent 58745e92d4
commit 11bfd0a838
21 changed files with 308 additions and 248 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
docs/config/config-dotfiles.md vendored
View File

@@ -14,11 +14,13 @@ Entry | Description
`ignoreempty` | If true, an empty template will not be deployed (defaults to the value of `ignoreempty`)
`instignore` | List of patterns to ignore when installing (enclose in quotes when using wildcards; see [ignore patterns](config-file.md#ignore-patterns))
`template` | If false, disable templating for this dotfile (defaults to the value of `template_dotfile_default`)
`trans_read` | Transformation key to apply when installing this dotfile (must be defined in the **trans_read** entry below; see [transformations](config-transformations.md))
`trans_write` | Transformation key to apply when updating this dotfile (must be defined in the **trans_write** entry below; see [transformations](config-transformations.md))
`trans_install` | Transformation key to apply when installing this dotfile (must be defined in the **trans_install** entry below; see [transformations](config-transformations.md))
`trans_update` | Transformation key to apply when updating this dotfile (must be defined in the **trans_update** entry below; see [transformations](config-transformations.md))
`upignore` | List of patterns to ignore when updating (enclose in quotes when using wildcards; see [ignore patterns](config-file.md#ignore-patterns))
<s>link_children</s> | Replaced by `link: link_children`
<s>trans</s> | Replaced by `trans_read`
<s>trans</s> | Replaced by `trans_install`
<s>trans_read</s> | Replaced by `trans_install`
<s>trans_write</s> | Replaced by `trans_update`
```yaml
<dotfile-key-name>:
@@ -37,8 +39,8 @@ Entry | Description
- <action-key>
template: (true|false)
chmod: '<file-permissions>'
trans_read: <transformation-key>
trans_write: <transformation-key>
trans_install: <transformation-key>
trans_update: <transformation-key>
```
## Dotfile actions

32
docs/config/config-transformations.md vendored
View File

@@ -14,14 +14,14 @@ For examples of transformation uses, see:
There are two types of transformations available:
* **Read transformations**: used to transform dotfiles before they are installed ([config](config-config.md) key `trans_read`)
* **Install transformations**: used to transform dotfiles before they are installed ([config](config-config.md) key `trans_install`)
* Used for commands `install` and `compare`
* They have two mandatory arguments:
* **{0}** will be replaced with the dotfile to process
* **{1}** will be replaced with a temporary file to store the result of the transformation
* This Happens **before** the dotfile is templated (see [templating](../template/templating.md))
* **Write transformations**: used to transform files before updating a dotfile ([config](config-config.md) key `trans_write`)
* **Update/Import transformations**: used to transform files before updating/importing a dotfile ([config](config-config.md) key `trans_update`)
* Used for command `update` and `import`
* They have two mandatory arguments:
* **{0}** will be replaced with the file path to update the dotfile with
@@ -36,13 +36,13 @@ Transformations also support additional positional arguments that must start fro
For example:
```yaml
trans_read:
trans_install:
targ: echo "$(basename {0}); {{@@ _dotfile_key @@}}; {2}; {3}" > {1}
dotfiles:
f_abc:
dst: /tmp/abc
src: abc
trans_read: targ "{{@@ profile @@}}" lastarg
trans_install: targ "{{@@ profile @@}}" lastarg
profiles:
p1:
dotfiles:
@@ -51,21 +51,21 @@ profiles:
will result in `abc; f_abc; p1; lastarg`.
## trans_read entry
## trans_install entry
The **trans_read** entry (optional) contains a transformations mapping (See [transformations](config-transformations.md)).
The **trans_install** entry (optional) contains a transformations mapping (See [transformations](config-transformations.md)).
```yaml
trans_read:
trans_install:
<trans-key>: <command-to-execute>
```
## trans_write entry
## trans_update entry
The **trans_write** entry (optional) contains a write transformations mapping (See [transformations](config-transformations.md)).
The **trans_update** entry (optional) contains a write transformations mapping (See [transformations](config-transformations.md)).
```yaml
trans_write:
trans_update:
<trans-key>: <command-to-execute>
```
@@ -77,10 +77,10 @@ and [template variables](../template/template-variables.md#template-variables)).
A very dumb example:
```yaml
trans_read:
trans_install:
r_echo_abs_src: echo "{0}: {{@@ _dotfile_abs_src @@}}" > {1}
r_echo_var: echo "{0}: {{@@ r_var @@}}" > {1}
trans_write:
trans_update:
w_echo_key: echo "{0}: {{@@ _dotfile_key @@}}" > {1}
w_echo_var: echo "{0}: {{@@ w_var @@}}" > {1}
variables:
@@ -90,11 +90,11 @@ dotfiles:
f_abc:
dst: ${tmpd}/abc
src: abc
trans_read: r_echo_abs_src
trans_write: w_echo_key
trans_install: r_echo_abs_src
trans_update: w_echo_key
f_def:
dst: ${tmpd}/def
src: def
trans_read: r_echo_var
trans_write: w_echo_var
trans_install: r_echo_var
trans_update: w_echo_var
```

24
docs/howto/sensitive-dotfiles.md vendored
View File

@@ -37,9 +37,9 @@ First you need to define the encryption/decryption methods, for example
```yaml
variables:
keyid: "11223344"
trans_read:
trans_install:
_decrypt: "gpg -q --for-your-eyes-only--no-tty -d {0} > {1}"
trans_write:
trans_update:
_encrypt: "gpg -q -r {{@@ keyid @@}} --armor --no-tty -o {1} -e {0}"
```
@@ -60,17 +60,17 @@ Using GPG keys:
```yaml
variables:
keyid: "11223344"
trans_read:
trans_install:
_decrypt: "gpg -q --for-your-eyes-only--no-tty -d {0} > {1}"
trans_write:
trans_update:
_encrypt: "gpg -q -r {{@@ keyid @@}} --armor --no-tty -o {1} -e {0}"
```
Passphrase is stored in an environment variable:
```yaml
trans_read:
trans_install:
_decrypt: "echo {{@@ env['THE_KEY'] @@}} | gpg -q --batch --yes --for-your-eyes-only --passphrase-fd 0 --no-tty -d {0} > {1}"
trans_write:
trans_update:
_encrypt: "echo {{@@ env['THE_KEY'] @@}} | gpg -q --batch --yes --passphrase-fd 0 --no-tty -o {1} -c {0}"
```
@@ -78,9 +78,9 @@ Passphrase is stored as a variable:
```yaml
variables:
gpg_password: "some password"
trans_read:
trans_install:
_decrypt: "echo {{@@ gpg_password @@}} | gpg -q --batch --yes --for-your-eyes-only --passphrase-fd 0 --no-tty -d {0} > {1}"
trans_write:
trans_update:
_encrypt: "echo {{@@ gpg_password @@}} | gpg -q --batch --yes --passphrase-fd 0 --no-tty -o {1} -c {0}"
```
@@ -88,9 +88,9 @@ Passphrase is retrieved using a script:
```yaml
dynvariables:
gpg_password: "./get-password.sh"
trans_read:
trans_install:
_decrypt: "echo {{@@ gpg_password @@}} | gpg -q --batch --yes --for-your-eyes-only --passphrase-fd 0 --no-tty -d {0} > {1}"
trans_write:
trans_update:
_encrypt: "echo {{@@ gpg_password @@}} | gpg -q --batch --yes --passphrase-fd 0 --no-tty -o {1} -c {0}"
```
@@ -100,9 +100,9 @@ variables:
gpg_password_file: "/tmp/the-password"
dynvariables:
gpg_password: "cat {{@@ gpg_password_file @@}}"
trans_read:
trans_install:
_decrypt: "echo {{@@ gpg_password @@}} | gpg -q --batch --yes --for-your-eyes-only --passphrase-fd 0 --no-tty -d {0} > {1}"
trans_write:
trans_update:
_encrypt: "echo {{@@ gpg_password @@}} | gpg -q --batch --yes --passphrase-fd 0 --no-tty -o {1} -c {0}"
```

6
docs/howto/store-compressed-directories.md vendored
View File

@@ -1,13 +1,13 @@
# Handle compressed directories
This is an example of how to use transformations (`trans_read` and `trans_write`) to store
This is an example of how to use transformations (`trans_install` and `trans_update`) to store
compressed directories and deploy them with dotdrop.
Start by defining the transformations:
```yaml
trans_read:
trans_install:
uncompress: "mkdir -p {1} && tar -xf {0} -C {1}"
trans_write:
trans_update:
compress: "tar -cf {1} -C {0} ."
```

73
dotdrop/cfg_aggregator.py
View File

@@ -69,23 +69,23 @@ class CfgAggregator:
return self.cfgyaml.del_dotfile_from_profile(dotfile.key, profile.key)
def new_dotfile(self, src, dst, link, chmod=None,
trans_read=None, trans_write=None):
trans_install=None, trans_update=None):
"""
import a new dotfile
@src: path in dotpath
@dst: path in FS
@link: LinkType
@chmod: file permission
@trans_read: read transformation
@trans_write: write transformation
@trans_install: read transformation
@trans_update: write transformation
"""
dst = self.path_to_dotfile_dst(dst)
dotfile = self.get_dotfile_by_src_dst(src, dst)
if not dotfile:
# add the dotfile
dotfile = self._create_new_dotfile(src, dst, link, chmod=chmod,
trans_read=trans_read,
trans_write=trans_write)
trans_install=trans_install,
trans_update=trans_update)
if not dotfile:
return False
@@ -237,25 +237,25 @@ class CfgAggregator:
########################################################
def _create_new_dotfile(self, src, dst, link, chmod=None,
trans_read=None, trans_write=None):
trans_install=None, trans_update=None):
"""create a new dotfile"""
# get a new dotfile with a unique key
key = self._get_new_dotfile_key(dst)
self.log.dbg(f'new dotfile key: {key}')
# add the dotfile
trans_r_key = trans_w_key = None
if trans_read:
trans_r_key = trans_read.key
if trans_write:
trans_w_key = trans_write.key
trans_install_key = trans_update_key = None
if trans_install:
trans_install_key = trans_install.key
if trans_update:
trans_update_key = trans_update.key
if not self.cfgyaml.add_dotfile(key, src, dst, link,
chmod=chmod,
trans_r_key=trans_r_key,
trans_w_key=trans_w_key):
trans_install_key=trans_install_key,
trans_update_key=trans_update_key):
return None
return Dotfile(key, dst, src,
trans_r=trans_read,
trans_w=trans_write)
trans_install=trans_install,
trans_update=trans_update)
########################################################
# parsing
@@ -297,15 +297,15 @@ class CfgAggregator:
self.actions = Action.parse_dict(self.cfgyaml.actions)
debug_list('actions', self.actions, self.debug)
# trans_r
self.log.dbg('parsing trans_r')
self.trans_r = Transform.parse_dict(self.cfgyaml.trans_r)
debug_list('trans_r', self.trans_r, self.debug)
# trans_install
self.log.dbg('parsing trans_install')
self.trans_install = Transform.parse_dict(self.cfgyaml.trans_install)
debug_list('trans_install', self.trans_install, self.debug)
# trans_w
self.log.dbg('parsing trans_w')
self.trans_w = Transform.parse_dict(self.cfgyaml.trans_w)
debug_list('trans_w', self.trans_w, self.debug)
# trans_update
self.log.dbg('parsing trans_update')
self.trans_update = Transform.parse_dict(self.cfgyaml.trans_update)
debug_list('trans_update', self.trans_update, self.debug)
# variables
self.log.dbg('parsing variables')
@@ -334,14 +334,17 @@ class CfgAggregator:
msg = f'default actions: {self.settings.default_actions}'
self.log.dbg(msg)
# patch trans_w/trans_r in dotfiles
# patch trans_install in dotfiles
trans_inst_args = self._get_trans_update_args(self.get_trans_install)
self._patch_keys_to_objs(self.dotfiles,
"trans_r",
self._get_trans_w_args(self.get_trans_r),
CfgYaml.key_trans_install,
trans_inst_args,
islist=False)
# patch trans_update in dotfiles
trans_update_args = self._get_trans_update_args(self.get_trans_update)
self._patch_keys_to_objs(self.dotfiles,
"trans_w",
self._get_trans_w_args(self.get_trans_w),
CfgYaml.key_trans_update,
trans_update_args,
islist=False)
self.log.dbg('done parsing cfgyaml into cfg_aggregator')
@@ -542,7 +545,7 @@ class CfgAggregator:
action = self._get_action(key)
return action
def _get_trans_w_args(self, getter):
def _get_trans_update_args(self, getter):
"""return transformation by key with the arguments"""
def getit(key):
fields = shlex.split(key)
@@ -557,16 +560,16 @@ class CfgAggregator:
return trans
return getit
def get_trans_r(self, key):
"""return the trans_r with this key"""
def get_trans_install(self, key):
"""return the trans_install with this key"""
try:
return next(x for x in self.trans_r if x.key == key)
return next(x for x in self.trans_install if x.key == key)
except StopIteration:
return None
def get_trans_w(self, key):
"""return the trans_w with this key"""
def get_trans_update(self, key):
"""return the trans_update with this key"""
try:
return next(x for x in self.trans_w if x.key == key)
return next(x for x in self.trans_update if x.key == key)
except StopIteration:
return None

146
dotdrop/cfg_yaml.py
View File

@@ -11,8 +11,8 @@ the upper layer:
* self.dotfiles
* self.profiles
* self.actions
* self.trans_r
* self.trans_w
* self.trans_install
* self.trans_update
* self.variables
Additionally a few methods are exported.
@@ -50,9 +50,11 @@ class CfgYaml:
key_dotfiles = 'dotfiles'
key_profiles = 'profiles'
key_actions = 'actions'
old_key_trans_r = 'trans'
key_trans_r = 'trans_read'
key_trans_w = 'trans_write'
old_key_trans = 'trans'
old_key_trans_r = 'trans_read'
old_key_trans_w = 'trans_write'
key_trans_install = 'trans_install'
key_trans_update = 'trans_update'
key_variables = 'variables'
key_dvariables = 'dynvariables'
key_uvariables = 'uservariables'
@@ -146,8 +148,8 @@ class CfgYaml:
self.dotfiles = {}
self.profiles = {}
self.actions = {}
self.trans_r = {}
self.trans_w = {}
self.trans_install = {}
self.trans_update = {}
self.variables = {}
if not os.path.exists(self._path):
@@ -248,10 +250,10 @@ class CfgYaml:
self.dotfiles = self._parse_blk_dotfiles(self._yaml_dict)
# parse the "actions" block
self.actions = self._parse_blk_actions(self._yaml_dict)
# parse the "trans_r" block
self.trans_r = self._parse_blk_trans_r(self._yaml_dict)
# parse the "trans_w" block
self.trans_w = self._parse_blk_trans_w(self._yaml_dict)
# parse the "trans_install" block
self.trans_install = self._parse_blk_trans_install(self._yaml_dict)
# parse the "trans_update" block
self.trans_update = self._parse_blk_trans_update(self._yaml_dict)
##################################################
# import elements
@@ -427,7 +429,7 @@ class CfgYaml:
return True
def add_dotfile(self, key, src, dst, link, chmod=None,
trans_r_key=None, trans_w_key=None):
trans_install_key=None, trans_update_key=None):
"""add a new dotfile"""
if key in self.dotfiles.keys():
return False
@@ -438,8 +440,8 @@ class CfgYaml:
self._dbg(f'new dotfile link: {link}')
if chmod:
self._dbg(f'new dotfile chmod: {chmod:o}')
self._dbg(f'new dotfile trans_r: {trans_r_key}')
self._dbg(f'new dotfile trans_w: {trans_w_key}')
self._dbg(f'new dotfile trans_install: {trans_install_key}')
self._dbg(f'new dotfile trans_update: {trans_update_key}')
# create the dotfile dict
df_dict = {
@@ -456,11 +458,11 @@ class CfgYaml:
if chmod:
df_dict[self.key_dotfile_chmod] = str(format(chmod, 'o'))
# trans_r/trans_w
if trans_r_key:
df_dict[self.key_trans_r] = str(trans_r_key)
if trans_w_key:
df_dict[self.key_trans_w] = str(trans_w_key)
# trans_install/trans_update
if trans_install_key:
df_dict[self.key_trans_install] = str(trans_install_key)
if trans_update_key:
df_dict[self.key_trans_update] = str(trans_update_key)
if self._debug:
self._dbg(f'dotfile dict: {df_dict}')
@@ -618,30 +620,25 @@ class CfgYaml:
self._debug_dict('actions block', actions)
return actions
def _parse_blk_trans_r(self, dic):
"""parse the "trans_r" block"""
key = self.key_trans_r
if self.old_key_trans_r in dic:
msg = '\"trans\" is deprecated, please use \"trans_read\"'
self._log.warn(msg)
dic[self.key_trans_r] = dic[self.old_key_trans_r]
del dic[self.old_key_trans_r]
trans_r = self._get_entry(dic, key, mandatory=False)
if trans_r:
trans_r = trans_r.copy()
def _parse_blk_trans_install(self, dic):
"""parse the "trans_install" block"""
trans_install = self._get_entry(dic, self.key_trans_install,
mandatory=False)
if trans_install:
trans_install = trans_install.copy()
if self._debug:
self._debug_dict('trans_r block', trans_r)
return trans_r
self._debug_dict('trans_install block', trans_install)
return trans_install
def _parse_blk_trans_w(self, dic):
"""parse the "trans_w" block"""
trans_w = self._get_entry(dic, self.key_trans_w,
mandatory=False)
if trans_w:
trans_w = trans_w.copy()
def _parse_blk_trans_update(self, dic):
"""parse the "trans_update" block"""
trans_update = self._get_entry(dic, self.key_trans_update,
mandatory=False)
if trans_update:
trans_update = trans_update.copy()
if self._debug:
self._debug_dict('trans_w block', trans_w)
return trans_w
self._debug_dict('trans_update block', trans_update)
return trans_update
def _parse_blk_variables(self, dic):
"""parse the "variables" block"""
@@ -817,7 +814,11 @@ class CfgYaml:
if not dotfiles:
return dotfiles
new = {}
for k, val in dotfiles.items():
# fix depreacated trans
self._fix_deprecated_trans_in_dict(val)
if self.key_dotfile_src not in val:
# add 'src' as key' if not present
val[self.key_dotfile_src] = k
@@ -825,14 +826,6 @@ class CfgYaml:
else:
new[k] = val
if self.old_key_trans_r in val:
# fix deprecated trans key
msg = f'{k} \"trans\" is deprecated, please use \"trans_read\"'
self._log.warn(msg)
val[self.key_trans_r] = val[self.old_key_trans_r]
del val[self.old_key_trans_r]
new[k] = val
if self.key_dotfile_link not in val:
# apply link value if undefined
value = self.settings[self.key_settings_link_dotfile_default]
@@ -1108,8 +1101,10 @@ class CfgYaml:
self.profiles = self._merge_dict(self.profiles, sub.profiles,
deep=True)
self.actions = self._merge_dict(self.actions, sub.actions)
self.trans_r = self._merge_dict(self.trans_r, sub.trans_r)
self.trans_w = self._merge_dict(self.trans_w, sub.trans_w)
self.trans_install = self._merge_dict(self.trans_install,
sub.trans_install)
self.trans_update = self._merge_dict(self.trans_update,
sub.trans_update)
self._clear_profile_vars(sub.variables)
self.imported_configs.append(path)
@@ -1189,6 +1184,53 @@ class CfgYaml:
return
self._fix_deprecated_link_by_default(yamldict)
self._fix_deprecated_dotfile_link(yamldict)
self._fix_deprecated_trans(yamldict)
def _fix_deprecated_trans_in_dict(self, yamldic):
# trans -> trans_install
old_key = self.old_key_trans
new_key = self.key_trans_install
if old_key in yamldic:
yamldic[old_key] = yamldic[new_key]
del yamldic[old_key]
msg = f'\"{old_key}\" is deprecated, '
msg += f', updated to {new_key}\"'
self._log.warn(msg)
self._dirty = True
self._dirty_deprecated = True
# trans_read -> trans_install
old_key = self.old_key_trans_r
new_key = self.key_trans_install
if old_key in yamldic:
yamldic[new_key] = yamldic[old_key]
del yamldic[old_key]
warn = f'deprecated \"{old_key}\"'
warn += f', updated to \"{new_key}\"'
self._log.warn(warn)
self._dirty = True
self._dirty_deprecated = True
# trans_write -> trans_update
old_key = self.old_key_trans_w
new_key = self.key_trans_update
if old_key in yamldic:
yamldic[new_key] = yamldic[old_key]
del yamldic[old_key]
warn = f'deprecated \"{old_key}\"'
warn += f', updated to \"{new_key}\"'
self._log.warn(warn)
self._dirty = True
self._dirty_deprecated = True
def _fix_deprecated_trans(self, yamldict):
"""fix deprecated trans key"""
if self.key_settings not in yamldict:
return
if not yamldict[self.key_settings]:
return
config = yamldict[self.key_settings]
self._fix_deprecated_trans_in_dict(config)
def _fix_deprecated_link_by_default(self, yamldict):
"""fix deprecated link_by_default"""
@@ -1786,8 +1828,8 @@ class CfgYaml:
self._debug_dict('entry dotfiles', self.dotfiles)
self._debug_dict('entry profiles', self.profiles)
self._debug_dict('entry actions', self.actions)
self._debug_dict('entry trans_r', self.trans_r)
self._debug_dict('entry trans_w', self.trans_w)
self._debug_dict('entry trans_install', self.trans_install)
self._debug_dict('entry trans_update', self.trans_update)
self._debug_dict('entry variables', self.variables)
def _debug_dict(self, title, elems):

25
dotdrop/dotdrop.py
View File

@@ -120,9 +120,10 @@ def _dotfile_compare(opts, dotfile, tmp):
# apply transformation
tmpsrc = None
if dotfile.trans_r:
if dotfile.trans_install:
LOG.dbg('applying transformation before comparing')
tmpsrc = apply_trans(opts.dotpath, dotfile, templ, debug=opts.debug)
tmpsrc = apply_install_trans(opts.dotpath, dotfile,
templ, debug=opts.debug)
if not tmpsrc:
# could not apply trans
return False
@@ -238,8 +239,9 @@ def _dotfile_install(opts, dotfile, tmpdir=None):
# nolink
src = dotfile.src
tmp = None
if dotfile.trans_r:
tmp = apply_trans(opts.dotpath, dotfile, templ, debug=opts.debug)
if dotfile.trans_install:
tmp = apply_install_trans(opts.dotpath, dotfile,
templ, debug=opts.debug)
if not tmp:
return False, dotfile.key, None
src = tmp
@@ -538,8 +540,8 @@ def cmd_importer(opts):
import_as=opts.import_as,
import_link=opts.import_link,
import_mode=opts.import_mode,
import_transw=opts.import_transw,
import_transr=opts.import_transr)
import_trans_install=opts.import_transr,
import_trans_update=opts.import_transw)
if tmpret < 0:
ret = False
elif tmpret > 0:
@@ -772,19 +774,20 @@ def _select(selections, dotfiles):
return selected
def apply_trans(dotpath, dotfile, templater, debug=False):
def apply_install_trans(dotpath, dotfile, templater, debug=False):
"""
apply the read transformation to the dotfile
apply the install transformation to the dotfile
return None if fails and new source if succeed
"""
src = dotfile.src
new_src = f'{src}.{TRANS_SUFFIX}'
trans = dotfile.trans_r
LOG.dbg(f'executing transformation: {trans}')
trans = dotfile.trans_install
LOG.dbg(f'executing install transformation: {trans}')
srcpath = os.path.join(dotpath, src)
temp = os.path.join(dotpath, new_src)
if not trans.transform(srcpath, temp, templater=templater, debug=debug):
msg = f'transformation \"{trans.key}\" failed for {dotfile.key}'
msg = f'install transformation \"{trans.key}\"'
msg += f'failed for {dotfile.key}'
LOG.err(msg)
if new_src and os.path.exists(new_src):
removepath(new_src, LOG)

54
dotdrop/dotfile.py
View File

@@ -14,12 +14,12 @@ class Dotfile(DictParser):
"""Represent a dotfile."""
# dotfile keys
key_noempty = 'ignoreempty'
key_trans_r = 'trans_read'
key_trans_w = 'trans_write'
key_trans_install = 'trans_install'
key_trans_update = 'trans_update'
key_template = 'template'
def __init__(self, key, dst, src,
actions=None, trans_r=None, trans_w=None,
actions=None, trans_install=None, trans_update=None,
link=LinkTypes.NOLINK, noempty=False,
cmpignore=None, upignore=None,
instignore=None, template=True, chmod=None,
@@ -30,8 +30,8 @@ class Dotfile(DictParser):
@dst: dotfile dst (in user's home usually)
@src: dotfile src (in dotpath)
@actions: dictionary of actions to execute for this dotfile
@trans_r: transformation to change dotfile before it is installed
@trans_w: transformation to change dotfile before updating it
@trans_install: transformation to change dotfile before it is installed
@trans_update: transformation to change dotfile before updating it
@link: link behavior
@noempty: ignore empty template if True
@upignore: patterns to ignore when updating
@@ -46,8 +46,8 @@ class Dotfile(DictParser):
self.link = LinkTypes.get(link)
self.noempty = noempty
self.src = src
self.trans_r = trans_r
self.trans_w = trans_w
self.trans_install = trans_install
self.trans_update = trans_update
self.upignore = upignore or []
self.cmpignore = cmpignore or []
self.instignore = instignore or []
@@ -57,14 +57,14 @@ class Dotfile(DictParser):
if self.link != LinkTypes.NOLINK and \
(
(trans_r and len(trans_r) > 0) or
(trans_w and len(trans_w) > 0)
(trans_install and len(trans_install) > 0) or
(trans_update and len(trans_update) > 0)
):
msg = f'[{key}] transformations disabled'
msg += ' because dotfile is linked'
self.log.warn(msg)
self.trans_r = []
self.trans_w = []
self.trans_install = []
self.trans_update = []
def get_dotfile_variables(self):
"""return this dotfile specific variables"""
@@ -83,25 +83,25 @@ class Dotfile(DictParser):
"""return all 'post' actions"""
return [a for a in self.actions if a.kind == Action.post]
def get_trans_r(self):
"""return trans_r object"""
return self.trans_r
def get_trans_install(self):
"""return trans_install object"""
return self.trans_install
def get_trans_w(self):
"""return trans_w object"""
return self.trans_w
def get_trans_update(self):
"""return trans_update object"""
return self.trans_update
@classmethod
def _adjust_yaml_keys(cls, value):
"""patch dict"""
value['noempty'] = value.get(cls.key_noempty, False)
value['trans_r'] = value.get(cls.key_trans_r)
value['trans_w'] = value.get(cls.key_trans_w)
value['trans_install'] = value.get(cls.key_trans_install)
value['trans_update'] = value.get(cls.key_trans_update)
value['template'] = value.get(cls.key_template, True)
# remove old entries
value.pop(cls.key_noempty, None)
value.pop(cls.key_trans_r, None)
value.pop(cls.key_trans_w, None)
value.pop(cls.key_trans_install, None)
value.pop(cls.key_trans_update, None)
return value
def __eq__(self, other):
@@ -116,6 +116,10 @@ class Dotfile(DictParser):
msg += f', dst:\"{self.dst}\"'
msg += f', link:\"{self.link}\"'
msg += f', template:{self.template}'
if self.trans_install:
msg += f', trans_install:{self.trans_install}'
if self.trans_update:
msg += f', trans_update:{self.trans_update}'
if self.chmod:
if isinstance(self.chmod, int) or len(self.chmod) == 3:
msg += f', chmod:{self.chmod:o}'
@@ -149,13 +153,13 @@ class Dotfile(DictParser):
for act in some:
out += f'\n{2*indent}- {act}'
out += f'\n{indent}trans_r:'
some = self.get_trans_r()
out += f'\n{indent}trans_install:'
some = self.get_trans_install()
if some:
out += f'\n{2*indent}- {some}'
out += f'\n{indent}trans_w:'
some = self.get_trans_w()
out += f'\n{indent}trans_update:'
some = self.get_trans_update()
if some:
out += f'\n{2*indent}- {some}'
return out

46
dotdrop/importer.py
View File

@@ -75,8 +75,8 @@ class Importer:
def import_path(self, path, import_as=None,
import_link=LinkTypes.NOLINK,
import_mode=False,
import_transw="",
import_transr=""):
import_trans_install="",
import_trans_update=""):
"""
import a dotfile pointed by path
returns:
@@ -90,24 +90,25 @@ class Importer:
self.log.err(f'\"{path}\" does not exist, ignored!')
return -1
# check transw if any
trans_write = None
trans_read = None
if import_transw:
trans_write = self.conf.get_trans_w(import_transw)
if import_transr:
trans_read = self.conf.get_trans_r(import_transr)
# check trans_update if any
trans_install = None
trans_update = None
if import_trans_install:
trans_install = self.conf.get_trans_install(import_trans_install)
if import_trans_update:
trans_update = self.conf.get_trans_update(import_trans_update)
return self._import(path, import_as=import_as,
import_link=import_link,
import_mode=import_mode,
trans_write=trans_write,
trans_read=trans_read)
trans_update=trans_update,
trans_install=trans_install)
def _import(self, path, import_as=None,
import_link=LinkTypes.NOLINK,
import_mode=False,
trans_write=None, trans_read=None):
trans_install=None,
trans_update=None):
"""
import path
returns:
@@ -162,17 +163,18 @@ class Importer:
self.log.dbg(f'import dotfile: src:{src} dst:{dst}')
if not self._import_to_dotpath(src, dst, trans_write=trans_write):
if not self._import_to_dotpath(src, dst, trans_update=trans_update):
return -1
return self._import_in_config(path, src, dst, perm, linktype,
import_mode,
trans_w=trans_write,
trans_r=trans_read)
trans_update=trans_update,
trans_install=trans_install)
def _import_in_config(self, path, src, dst, perm,
linktype, import_mode,
trans_r=None, trans_w=None):
trans_install=None,
trans_update=None):
"""
import path
returns:
@@ -190,8 +192,8 @@ class Importer:
# add file to config file
retconf = self.conf.new_dotfile(src, dst, linktype, chmod=chmod,
trans_read=trans_r,
trans_write=trans_w)
trans_install=trans_install,
trans_update=trans_update)
if not retconf:
self.log.warn(f'\"{path}\" ignored during import')
return 0
@@ -222,7 +224,7 @@ class Importer:
self.log.dbg('will overwrite existing file')
return True
def _import_to_dotpath(self, in_dotpath, in_fs, trans_write=None):
def _import_to_dotpath(self, in_dotpath, in_fs, trans_update=None):
"""
prepare hierarchy for dotfile in dotpath and copy file
"""
@@ -237,8 +239,8 @@ class Importer:
self.log.dry(f'would copy {in_fs} to {srcf}')
return True
# apply trans_w
in_fs = self._apply_trans_w(in_fs, trans_write)
# apply trans_update
in_fs = self._apply_trans_update(in_fs, trans_update)
if not in_fs:
# transformation failed
return False
@@ -290,7 +292,7 @@ class Importer:
return True
return False
def _apply_trans_w(self, path, trans):
def _apply_trans_update(self, path, trans):
"""
apply transformation to path on filesystem)
returns

8
dotdrop/options.py
View File

@@ -92,8 +92,8 @@ Options:
-p --profile=<profile> Specify the profile to use [default: {PROFILE}].
-P --show-patch Provide a one-liner to manually patch template.
-s --as=<path> Import as a different path from actual path.
--transr=<key> Associate trans_read key on import.
--transw=<key> Apply trans_write key on import.
--transr=<key> Associate trans_install key on import.
--transw=<key> Apply trans_update key on import.
-t --temp Install to a temporary directory for review.
-T --template Only template dotfiles.
-V --verbose Be verbose.
@@ -318,8 +318,8 @@ class Options(AttrMonitor):
self.import_ignore.extend(self.impignore)
self.import_ignore.append(f'*{self.install_backup_suffix}')
self.import_ignore = uniq_list(self.import_ignore)
self.import_transw = self.args['--transw']
self.import_transr = self.args['--transr']
self.import_trans_install = self.args['--transr']
self.import_trans_update = self.args['--transw']
def _apply_args_update(self):
"""update specifics"""

6
dotdrop/updater.py
View File

@@ -122,7 +122,7 @@ class Updater:
return True
# apply write transformation if any
new_path = self._apply_trans_w(deployed_path, dotfile)
new_path = self._apply_trans_update(deployed_path, dotfile)
if not new_path:
return False
@@ -150,9 +150,9 @@ class Updater:
removepath(new_path, logger=self.log)
return ret
def _apply_trans_w(self, path, dotfile):
def _apply_trans_update(self, path, dotfile):
"""apply write transformation to dotfile"""
trans = dotfile.get_trans_w()
trans = dotfile.get_trans_update()
if not trans:
return path
self.log.dbg(f'executing write transformation {trans}')

4
manpage/dotdrop.1 vendored
View File

@@ -105,11 +105,11 @@ Import as a different path from actual path.
.TP
.B
\fB--transr\fP=<key>
Associate trans_read key on import.
Associate trans_install key on import.
.TP
.B
\fB--transw\fP=<key>
Apply trans_write key on import.
Apply trans_update key on import.
.RE
.TP
.B

4
manpage/dotdrop.txt2man vendored
View File

@@ -39,8 +39,8 @@ COMMANDS
-m --preserve-mode Insert a chmod entry in the dotfile with its mode.
-p --profile=<profile> Specify the profile to use.
-s --as=<path> Import as a different path from actual path.
--transr=<key> Associate trans_read key on import.
--transw=<key> Apply trans_write key on import.
--transr=<key> Associate trans_install key on import.
--transw=<key> Apply trans_update key on import.
compare Compare dotfiles
-C --file=<path> Path of dotfile to compare.

48
tests-ng/import-with-trans.sh vendored
View File

@@ -34,19 +34,21 @@ echo "dotfiles source (dotpath): ${tmps}"
# the dotfile destination
tmpd=$(mktemp -d --suffix='-dotdrop-tests' || mktemp -d)
echo "dotfiles destination: ${tmpd}"
tmptmp=$(mktemp -d --suffix='-dotdrop-tests' || mktemp -d)
clear_on_exit "${tmps}"
clear_on_exit "${tmpd}"
clear_on_exit "${tmptmp}"
# create the config file
cfg="${tmps}/config.yaml"
cat > "${cfg}" << _EOF
trans_read:
trans_install:
base64: "cat {0} | base64 -d > {1}"
decompress: "mkdir -p {1} && tar -xf {0} -C {1}"
decrypt: "echo {{@@ profile @@}} | gpg -q --batch --yes --passphrase-fd 0 --no-tty -d {0} > {1}"
trans_write:
trans_update:
base64: "cat {0} | base64 > {1}"
compress: "tar -cf {1} -C {0} ."
encrypt: "echo {{@@ profile @@}} | gpg -q --batch --yes --passphrase-fd 0 --no-tty -o {1} -c {0}"
@@ -90,16 +92,16 @@ cd "${ddpath}" | ${bin} import -f -c "${cfg}" -p p1 -b -V --transw=encrypt --tra
# check content in dotpath
echo "checking content"
file "${tmps}"/dotfiles/"${tmpd}"/abc | grep -i 'text'
cat "${tmpd}"/abc | base64 > "${tmps}"/test-abc
diff "${tmps}"/dotfiles/"${tmpd}"/abc "${tmps}"/test-abc
cat "${tmpd}"/abc | base64 > "${tmptmp}"/test-abc
diff "${tmps}"/dotfiles/"${tmpd}"/abc "${tmptmp}"/test-abc
file "${tmps}"/dotfiles/"${tmpd}"/def | grep -i 'tar'
tar -cf "${tmps}"/test-def -C "${tmpd}"/def .
diff "${tmps}"/dotfiles/"${tmpd}"/def "${tmps}"/test-def
tar -cf "${tmptmp}"/test-def -C "${tmpd}"/def .
diff "${tmps}"/dotfiles/"${tmpd}"/def "${tmptmp}"/test-def
file "${tmps}"/dotfiles/"${tmpd}"/ghi | grep -i 'gpg symmetrically encrypted data\|PGP symmetric key encrypted data'
echo p1 | gpg -q --batch --yes --passphrase-fd 0 --no-tty -d "${tmps}"/dotfiles/"${tmpd}"/ghi > "${tmps}"/test-ghi
diff "${tmps}"/test-ghi "${tmpd}"/ghi
echo p1 | gpg -q --batch --yes --passphrase-fd 0 --no-tty -d "${tmps}"/dotfiles/"${tmpd}"/ghi > "${tmptmp}"/test-ghi
diff "${tmptmp}"/test-ghi "${tmpd}"/ghi
# check is imported in config
echo "checking imported in config"
@@ -108,33 +110,33 @@ cd "${ddpath}" | ${bin} -p p1 -c "${cfg}" files | grep '^f_abc'
cd "${ddpath}" | ${bin} -p p1 -c "${cfg}" files | grep '^d_def'
cd "${ddpath}" | ${bin} -p p1 -c "${cfg}" files | grep '^f_ghi'
# check has trans_write and trans_read in config
echo "checking trans_write is set in config"
# check has trans_update and trans_install in config
echo "checking trans_update is set in config"
echo "--------------"
cat "${cfg}"
echo "--------------"
cat "${cfg}" | grep -A 4 'f_abc:' | grep 'trans_write: base64'
cat "${cfg}" | grep -A 4 'd_def:' | grep 'trans_write: compress'
cat "${cfg}" | grep -A 4 'f_ghi:' | grep 'trans_write: encrypt'
cat "${cfg}" | grep -A 4 'f_abc:' | grep 'trans_update: base64'
cat "${cfg}" | grep -A 4 'd_def:' | grep 'trans_update: compress'
cat "${cfg}" | grep -A 4 'f_ghi:' | grep 'trans_update: encrypt'
cat "${cfg}" | grep -A 4 'f_abc:' | grep 'trans_read: base64'
cat "${cfg}" | grep -A 4 'd_def:' | grep 'trans_read: decompress'
cat "${cfg}" | grep -A 4 'f_ghi:' | grep 'trans_read: decrypt'
cat "${cfg}" | grep -A 4 'f_abc:' | grep 'trans_install: base64'
cat "${cfg}" | grep -A 4 'd_def:' | grep 'trans_install: decompress'
cat "${cfg}" | grep -A 4 'f_ghi:' | grep 'trans_install: decrypt'
# install these
echo "install and check"
rm "${tmpd}"/abc
rm -r "${tmpd}"/def
rm "${tmpd}"/ghi
rm -rf "${tmpd:?}"/*
cd "${ddpath}" | ${bin} install -f -c "${cfg}" -p p1 -b -V
# test exist
echo "check exist"
[ ! -e "${tmpd}"/abc ] && exit 1
[ ! -d "${tmpd}"/def/a ] && exit 1
[ ! -e "${tmpd}"/def/a/file ] && exit 1
[ ! -e "${tmpd}"/ghi ] && exit 1
cat "${cfg}"
tree "${tmpd}"
[ ! -e "${tmpd}"/abc ] && echo "${tmpd}/abc does not exist" && exit 1
[ ! -d "${tmpd}"/def/a ] && echo "${tmpd}/def/a does not exist" && exit 1
[ ! -e "${tmpd}"/def/a/file ] && echo "${tmpd}/def/a/file does not exist" && exit 1
[ ! -e "${tmpd}"/ghi ] && echo "${tmpd}/ghi does not exist" && exit 1
# test content
echo "check content"

6
tests-ng/tests_launcher.py
View File

@@ -108,8 +108,10 @@ def run_tests(max_jobs=None, stop_on_first_err=True, with_spinner=True):
failed += 1
print()
if stop_on_first_err:
print(log_out)
print(log_err)
if log_out:
print(log_out)
if log_err:
print(log_err)
print(f'test \"{name}\" failed ({ret}): {reason}')
if stop_on_first_err:
ex.shutdown(wait=False)

6
tests/helpers.py
View File

@@ -246,7 +246,7 @@ def create_yaml_keyval(pairs, parent_dir=None, top_key=None):
# pylint: disable=W0102
def populate_fake_config(config, dotfiles={}, profiles={}, actions={},
trans={}, trans_write={}, variables={},
trans_install={}, trans_update={}, variables={},
dynvariables={}):
"""Adds some juicy content to config files"""
is_path = isinstance(config, str)
@@ -257,8 +257,8 @@ def populate_fake_config(config, dotfiles={}, profiles={}, actions={},
config['dotfiles'] = dotfiles
config['profiles'] = profiles
config['actions'] = actions
config['trans_read'] = trans
config['trans_write'] = trans_write
config['trans_install'] = trans_install
config['trans_update'] = trans_update
config['variables'] = variables
config['dynvariables'] = dynvariables

16
tests/test_import.py
View File

@@ -239,10 +239,10 @@ class TestImport(unittest.TestCase):
},
'a_log_ed': 'echo 2',
},
'trans': {
'trans_install': {
't_log_ed': 'echo 3',
},
'trans_write': {
'trans_update': {
'tw_log_ed': 'echo 4',
},
'variables': {
@@ -273,10 +273,10 @@ class TestImport(unittest.TestCase):
},
'a_log_ing': 'echo a',
},
'trans': {
'trans_install': {
't_log_ing': 'echo b',
},
'trans_write': {
'trans_update': {
'tw_log_ing': 'echo c',
},
'variables': {
@@ -352,10 +352,10 @@ class TestImport(unittest.TestCase):
self.assertFalse(any(a.endswith('ing') for a in actions))
# testing transformations
transformations = ycont['trans_read'].keys()
transformations = ycont['trans_install'].keys()
self.assertTrue(all(t.endswith('ed') for t in transformations))
self.assertFalse(any(t.endswith('ing') for t in transformations))
transformations = ycont['trans_write'].keys()
transformations = ycont['trans_update'].keys()
self.assertTrue(all(t.endswith('ed') for t in transformations))
self.assertFalse(any(t.endswith('ing') for t in transformations))
@@ -394,10 +394,10 @@ class TestImport(unittest.TestCase):
self.assertFalse(any(action.endswith('ed') for action in actions))
# testing transformations
transformations = ycont['trans_read'].keys()
transformations = ycont['trans_install'].keys()
self.assertTrue(all(t.endswith('ing') for t in transformations))
self.assertFalse(any(t.endswith('ed') for t in transformations))
transformations = ycont['trans_write'].keys()
transformations = ycont['trans_update'].keys()
self.assertTrue(all(t.endswith('ing') for t in transformations))
self.assertFalse(any(t.endswith('ed') for t in transformations))

8
tests/test_install.py
View File

@@ -46,9 +46,9 @@ def fake_config(path, dotfiles, profile,
file.write(' actions:\n')
for action in dotfile.actions:
file.write(f' - {action.key}\n')
if dotfile.trans_r:
for trans in dotfile.trans_r:
file.write(f' trans_read: {trans.key}\n')
if dotfile.trans_install:
for trans in dotfile.trans_install:
file.write(f' trans_install: {trans.key}\n')
file.write('profiles:\n')
file.write(f' {profile}:\n')
file.write(' dotfiles:\n')
@@ -174,7 +174,7 @@ exec bspwm
fcontent9, _ = create_random_file(tmp, content=trans1)
dst9 = os.path.join(dst, get_string(6))
dotfile9 = Dotfile(get_string(6), dst9, os.path.basename(fcontent9),
trans_r=[the_trans])
trans_install=[the_trans])
# to test template
f10, _ = create_random_file(tmp, content='{{@@ header() @@}}')

2
tests/test_misc.py
View File

@@ -127,7 +127,7 @@ class TestImporter(unittest.TestCase):
path, _ = create_random_file(tmpdir)
imp = Importer('profile', None, '', '', {})
self.assertEqual(imp._apply_trans_w(path, trans), None)
self.assertEqual(imp._apply_trans_update(path, trans), None)
class TestActions(unittest.TestCase):

2
tests/test_update.py
View File

@@ -121,7 +121,7 @@ class TestUpdate(unittest.TestCase):
# retrieve the path of the sub in the dotpath
d1indotpath = os.path.join(opt.dotpath, dotfile.src)
d1indotpath = os.path.expanduser(d1indotpath)
dotfile.trans_w = trans
dotfile.trans_update = trans
# update template
opt.update_path = [d3t]

28
tests/test_yamlcfg.py
View File

@@ -298,10 +298,10 @@ profiles:
},
'a_log_ed': 'echo 2',
},
'trans': {
'trans_install': {
't_log_ed': 'echo 3',
},
'trans_write': {
'trans_update': {
'tw_log_ed': 'echo 4',
},
'variables': {
@@ -335,10 +335,10 @@ profiles:
},
'a_log_ing': 'echo a',
},
'trans': {
'trans_install': {
't_log_ing': 'echo b',
},
'trans_write': {
'trans_update': {
'tw_log_ing': 'echo c',
},
'variables': {
@@ -406,8 +406,8 @@ profiles:
self.assert_is_subset(post_ed, post_ing)
# test transactions
self.assert_is_subset(imported_cfg.trans_r, importing_cfg.trans_r)
self.assert_is_subset(imported_cfg.trans_w, importing_cfg.trans_w)
self.assert_is_subset(imported_cfg.trans_install, importing_cfg.trans_install)
self.assert_is_subset(imported_cfg.trans_update, importing_cfg.trans_update)
# test variables
imported_vars = {
@@ -504,10 +504,10 @@ profiles:
},
'a_log': 'echo 2',
},
'trans': {
'trans_install': {
't_log': 'echo 3',
},
'trans_write': {
'trans_update': {
'tw_log': 'echo 4',
},
'variables': {
@@ -542,10 +542,10 @@ profiles:
},
'a_log': 'echo a',
},
'trans': {
'trans_install': {
't_log': 'echo b',
},
'trans_write': {
'trans_update': {
'tw_log': 'echo c',
},
'variables': {
@@ -605,12 +605,12 @@ profiles:
# test transactions
self.assertFalse(any(
imported_cfg.trans_r[key] == importing_cfg.trans_r[key]
for key in imported_cfg.trans_r
imported_cfg.trans_install[key] == importing_cfg.trans_install[key]
for key in imported_cfg.trans_install
))
self.assertFalse(any(
imported_cfg.trans_w[key] == importing_cfg.trans_w[key]
for key in imported_cfg.trans_w
imported_cfg.trans_update[key] == importing_cfg.trans_update[key]
for key in imported_cfg.trans_update
))
# test variables