mirror of
https://github.com/deadc0de6/dotdrop.git
synced 2026-02-04 15:39:43 +00:00
add import and trans{_r,_w}
This commit is contained in:
@@ -16,14 +16,14 @@ There are two types of transformations available:
|
|||||||
|
|
||||||
* **Read transformations**: used to transform dotfiles before they are installed ([config](config-config.md) key `trans_read`)
|
* **Read transformations**: used to transform dotfiles before they are installed ([config](config-config.md) key `trans_read`)
|
||||||
* Used for commands `install` and `compare`
|
* Used for commands `install` and `compare`
|
||||||
* They have two arguments:
|
* They have two mandatory arguments:
|
||||||
* **{0}** will be replaced with the dotfile to process
|
* **{0}** will be replaced with the dotfile to process
|
||||||
* **{1}** will be replaced with a temporary file to store the result of the transformation
|
* **{1}** will be replaced with a temporary file to store the result of the transformation
|
||||||
* This Happens **before** the dotfile is templated (see [templating](templating.md))
|
* This Happens **before** the dotfile is templated (see [templating](templating.md))
|
||||||
|
|
||||||
* **Write transformations**: used to transform files before updating a dotfile ([config](config-config.md) key `trans_write`)
|
* **Write transformations**: used to transform files before updating a dotfile ([config](config-config.md) key `trans_write`)
|
||||||
* Used for command `update`
|
* Used for command `update` and `import`
|
||||||
* They have two arguments:
|
* They have two mandatory arguments:
|
||||||
* **{0}** will be replaced with the file path to update the dotfile with
|
* **{0}** will be replaced with the file path to update the dotfile with
|
||||||
* **{1}** will be replaced with a temporary file to store the result of the transformation
|
* **{1}** will be replaced with a temporary file to store the result of the transformation
|
||||||
|
|
||||||
|
|||||||
@@ -1,70 +1,109 @@
|
|||||||
# Handle secrets
|
# Handle secrets
|
||||||
|
|
||||||
Two solutions exist, the first one using an unversioned file (see [Environment variables](../templating.md#environment-variables))
|
* [Using environment variables](#using-environment-variables)
|
||||||
and the second using transformations (see [Store encrypted dotfiles](#store-encrypted-dotfiles)).
|
* [Store encrypted dotfiles using GPG](#store-encrypted-dotfiles-using-gpg)
|
||||||
|
* [GPG examples](#gpg-examples)
|
||||||
|
|
||||||
* [Store encrypted dotfiles](#store-encrypted-dotfiles)
|
## Using environment variables
|
||||||
* [Load passphrase from file](#load-passphrase-from-file)
|
|
||||||
|
|
||||||
## Store encrypted dotfiles
|
For example, you can have an `.env` file in the directory where your `config.yaml` lies:
|
||||||
|
```bash
|
||||||
|
## Some secrets
|
||||||
|
pass="verysecurepassword"
|
||||||
|
```
|
||||||
|
|
||||||
Here's an example of part of a config file to use gpg encrypted dotfiles:
|
If this file contains secrets that should not be tracked by Git,
|
||||||
|
put it in your `.gitignore`.
|
||||||
|
|
||||||
|
You can then invoke dotdrop with the help of an alias
|
||||||
|
```bash
|
||||||
|
# when dotdrop is installed as a submodule
|
||||||
|
alias dotdrop='eval $(grep -v "^#" ~/dotfiles/.env) ~/dotfiles/dotdrop.sh'
|
||||||
|
|
||||||
|
# when dotdrop is installed from package
|
||||||
|
alias dotdrop='eval $(grep -v "^#" ~/dotfiles/.env) /usr/bin/dotdrop --cfg=~/dotfiles/config.yaml'
|
||||||
|
```
|
||||||
|
|
||||||
|
The above aliases load all the variables from `~/dotfiles/.env`
|
||||||
|
(while omitting lines starting with `#`) before calling dotdrop.
|
||||||
|
Defined variables can then be used [in the config](../config-file.md#template-config-entries)
|
||||||
|
or [for templating dotfiles](../templating.md)
|
||||||
|
|
||||||
|
For more see [the doc on environment variables](../templating.md#environment-variables).
|
||||||
|
|
||||||
|
## Store encrypted dotfiles using GPG
|
||||||
|
|
||||||
|
First you need to define the encryption/decryption methods, for example
|
||||||
```yaml
|
```yaml
|
||||||
dotfiles:
|
variables:
|
||||||
f_secret:
|
keyid: "11223344"
|
||||||
dst: ~/.secret
|
|
||||||
src: secret
|
|
||||||
trans_read: _gpg
|
|
||||||
trans_read:
|
trans_read:
|
||||||
_gpg: gpg2 -q --for-your-eyes-only --no-tty -d {0} > {1}
|
_decrypt: "gpg -q --for-your-eyes-only--no-tty -d {0} > {1}"
|
||||||
|
trans_write:
|
||||||
|
_encrypt: "gpg -q -r {{@@ keyid @@}} --armor --no-tty -o {1} -e {0}"
|
||||||
```
|
```
|
||||||
|
|
||||||
The above config allows to store the dotfile `~/.secret` encrypted in the *dotpath*
|
You can then import your dotfile and specify the transformations to apply/associate.
|
||||||
directory and uses gpg to decrypt it when `install` is run.
|
|
||||||
|
|
||||||
Here's how to deploy the above solution:
|
|
||||||
|
|
||||||
* Import the clear dotfile (what creates the correct entries in the config file):
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
$ dotdrop import ~/.secret
|
dotdrop import --transw=_encrypt --transr=_decrypt ~/.secret
|
||||||
```
|
```
|
||||||
|
|
||||||
* Encrypt the original dotfile:
|
Now whenever you install/compare your dotfile, the `_decrypt` transformation will be executed
|
||||||
|
to get the clear version of the file.
|
||||||
```bash
|
When updating the `_encrypt` transformation will transform the file to store it encrypted.
|
||||||
$ <some-gpg-command> ~/.secret
|
|
||||||
```
|
|
||||||
|
|
||||||
* Overwrite the dotfile with the encrypted version:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
$ cp <encrypted-version-of-secret> dotfiles/secret
|
|
||||||
```
|
|
||||||
|
|
||||||
* Edit the config file and add the transformation to the dotfile
|
|
||||||
(as shown in the example above)
|
|
||||||
|
|
||||||
* Commit and push the changes
|
|
||||||
|
|
||||||
See [transformations](../config-transformations.md).
|
See [transformations](../config-transformations.md).
|
||||||
|
|
||||||
## Load passphrase from file
|
## gpg examples
|
||||||
|
|
||||||
|
Using GPG keys:
|
||||||
|
```yaml
|
||||||
|
variables:
|
||||||
|
keyid: "11223344"
|
||||||
|
trans_read:
|
||||||
|
_decrypt: "gpg -q --for-your-eyes-only--no-tty -d {0} > {1}"
|
||||||
|
trans_write:
|
||||||
|
_encrypt: "gpg -q -r {{@@ keyid @@}} --armor --no-tty -o {1} -e {0}"
|
||||||
|
```
|
||||||
|
|
||||||
|
Passphrase is stored in a environement variable:
|
||||||
|
```yaml
|
||||||
|
trans_read:
|
||||||
|
_decrypt: "echo {{@@ env['THE_KEY'] @@}} | gpg -q --batch --yes --for-your-eyes-only --passphrase-fd 0 --no-tty -d {0} > {1}"
|
||||||
|
trans_write:
|
||||||
|
_encrypt: "echo {{@@ env['THE_KEY'] @@}} | gpg -q --batch --yes --passphrase-fd 0 --no-tty -o {1} -c {0}"
|
||||||
|
```
|
||||||
|
|
||||||
|
Passphrase is stored as a variable:
|
||||||
|
```yaml
|
||||||
|
variables:
|
||||||
|
gpg_password: "some password"
|
||||||
|
trans_read:
|
||||||
|
_decrypt: "echo {{@@ gpg_password @@}} | gpg -q --batch --yes --for-your-eyes-only --passphrase-fd 0 --no-tty -d {0} > {1}"
|
||||||
|
trans_write:
|
||||||
|
_encrypt: "echo {{@@ gpg_password @@}} | gpg -q --batch --yes --passphrase-fd 0 --no-tty -o {1} -c {0}"
|
||||||
|
```
|
||||||
|
|
||||||
Passphrase is retrieved using a script:
|
Passphrase is retrieved using a script:
|
||||||
```yaml
|
```yaml
|
||||||
variables:
|
dynvariables:
|
||||||
gpg_password: "./get-password.sh"
|
gpg_password: "./get-password.sh"
|
||||||
trans_read:
|
trans_read:
|
||||||
_gpg: "gpg2 --batch --yes --passphrase-file <({{@@ gpg_password @@}}) -q --for-your-eyes-only --no-tty -d {0} > {1}"
|
_decrypt: "echo {{@@ gpg_password @@}} | gpg -q --batch --yes --for-your-eyes-only --passphrase-fd 0 --no-tty -d {0} > {1}"
|
||||||
|
trans_write:
|
||||||
|
_encrypt: "echo {{@@ gpg_password @@}} | gpg -q --batch --yes --passphrase-fd 0 --no-tty -o {1} -c {0}"
|
||||||
```
|
```
|
||||||
|
|
||||||
Passphrase is stored in a file directly:
|
Passphrase is stored in a file:
|
||||||
```yaml
|
```yaml
|
||||||
variables:
|
variables:
|
||||||
gpg_password_file: "/tmp/the-password"
|
gpg_password_file: "/tmp/the-password"
|
||||||
|
dynvariables:
|
||||||
|
gpg_password: "cat {{@@ gpg_password_file @@}}"
|
||||||
trans_read:
|
trans_read:
|
||||||
_gpg: "gpg2 --batch --yes --passphrase-file <(cat {{@@ gpg_password_file @@}}) -q --for-your-eyes-only --no-tty -d {0} > {1}"
|
_decrypt: "echo {{@@ gpg_password @@}} | gpg -q --batch --yes --for-your-eyes-only --passphrase-fd 0 --no-tty -d {0} > {1}"
|
||||||
|
trans_write:
|
||||||
|
_encrypt: "echo {{@@ gpg_password @@}} | gpg -q --batch --yes --passphrase-fd 0 --no-tty -o {1} -c {0}"
|
||||||
```
|
```
|
||||||
|
|
||||||
See [transformations](../config-transformations.md).
|
See also [transformations](../config-transformations.md).
|
||||||
@@ -3,34 +3,27 @@
|
|||||||
This is an example of how to use transformations (`trans_read` and `trans_write`) to store
|
This is an example of how to use transformations (`trans_read` and `trans_write`) to store
|
||||||
compressed directories and deploy them with dotdrop.
|
compressed directories and deploy them with dotdrop.
|
||||||
|
|
||||||
Config file:
|
Start by defining the transformations:
|
||||||
```yaml
|
```yaml
|
||||||
trans_read:
|
trans_read:
|
||||||
uncompress: "mkdir -p {1} && tar -xf {0} -C {1}"
|
uncompress: "mkdir -p {1} && tar -xf {0} -C {1}"
|
||||||
trans_write:
|
trans_write:
|
||||||
compress: "tar -cf {1} -C {0} ."
|
compress: "tar -cf {1} -C {0} ."
|
||||||
config:
|
|
||||||
backup: true
|
|
||||||
create: true
|
|
||||||
dotpath: dotfiles
|
|
||||||
dotfiles:
|
|
||||||
d_somedir:
|
|
||||||
dst: ~/.somedir
|
|
||||||
src: somedir
|
|
||||||
trans_read: uncompress
|
|
||||||
trans_write: compress
|
|
||||||
profiles:
|
|
||||||
p1:
|
|
||||||
dotfiles:
|
|
||||||
- d_somedir
|
|
||||||
```
|
```
|
||||||
|
|
||||||
The *read* transformation `uncompress` is used to execute the below command before deploying the dotfile (where `{0}` is the source and `{1}` the destination):
|
Then import the directory by specifying which transformations to apply/associate:
|
||||||
|
```bash
|
||||||
|
dotdrop import --transw=compress --transr=uncompress ~/.somedir
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The *read* transformation `uncompress` is used to execute the below command before installing/comparing the dotfile (where `{0}` is the source and `{1}` the destination):
|
||||||
|
```bash
|
||||||
mkdir -p {1} && tar -xf {0} -C {1}
|
mkdir -p {1} && tar -xf {0} -C {1}
|
||||||
```
|
```
|
||||||
|
|
||||||
And the *write* transformation `compress` is run when updating the dotfile directory by compressing it (where `{0}` is the source and `{1}` the destination):
|
And the *write* transformation `compress` is run when updating the dotfile directory by compressing it (where `{0}` is the source and `{1}` the destination):
|
||||||
```
|
```bash
|
||||||
tar -cf {1} -C {0} .
|
tar -cf {1} -C {0} .
|
||||||
```
|
```
|
||||||
|
|
||||||
|
See [transformations](../config-transformations.md).
|
||||||
@@ -75,27 +75,7 @@ It's possible to access environment variables inside the templates:
|
|||||||
This allows for storing host-specific properties and/or secrets in environment variables.
|
This allows for storing host-specific properties and/or secrets in environment variables.
|
||||||
It is recommended to use `variables` (see [config variables](config-file.md#variables))
|
It is recommended to use `variables` (see [config variables](config-file.md#variables))
|
||||||
instead of environment variables unless these contain sensitive information that
|
instead of environment variables unless these contain sensitive information that
|
||||||
shouldn't be versioned in Git.
|
shouldn't be versioned in Git (see [handle secrets doc](howto/sensitive-dotfiles.md)).
|
||||||
|
|
||||||
For example, you can have an `.env` file in the directory where your `config.yaml` lies:
|
|
||||||
```
|
|
||||||
## Some secrets
|
|
||||||
pass="verysecurepassword"
|
|
||||||
```
|
|
||||||
If this file contains secrets that should not be tracked by Git,
|
|
||||||
put it in your `.gitignore`.
|
|
||||||
|
|
||||||
You can then invoke dotdrop with the help of an alias
|
|
||||||
```bash
|
|
||||||
# when dotdrop is installed as a submodule
|
|
||||||
alias dotdrop='eval $(grep -v "^#" ~/dotfiles/.env) ~/dotfiles/dotdrop.sh'
|
|
||||||
|
|
||||||
# when dotdrop is installed from pypi or aur
|
|
||||||
alias dotdrop='eval $(grep -v "^#" ~/dotfiles/.env) /usr/bin/dotdrop --cfg=~/dotfiles/config.yaml'
|
|
||||||
```
|
|
||||||
|
|
||||||
The above aliases load all the variables from `~/dotfiles/.env`
|
|
||||||
(while omitting lines starting with `#`) before calling dotdrop.
|
|
||||||
|
|
||||||
## Template methods
|
## Template methods
|
||||||
|
|
||||||
|
|||||||
@@ -56,28 +56,36 @@ class CfgAggregator:
|
|||||||
"""remove this dotfile from this profile"""
|
"""remove this dotfile from this profile"""
|
||||||
return self.cfgyaml.del_dotfile_from_profile(dotfile.key, profile.key)
|
return self.cfgyaml.del_dotfile_from_profile(dotfile.key, profile.key)
|
||||||
|
|
||||||
def new_dotfile(self, src, dst, link, chmod=None):
|
def new_dotfile(self, src, dst, link, chmod=None,
|
||||||
|
trans_read=None, trans_write=None):
|
||||||
"""
|
"""
|
||||||
import a new dotfile
|
import a new dotfile
|
||||||
@src: path in dotpath
|
@src: path in dotpath
|
||||||
@dst: path in FS
|
@dst: path in FS
|
||||||
@link: LinkType
|
@link: LinkType
|
||||||
@chmod: file permission
|
@chmod: file permission
|
||||||
|
@trans_read: read transformation
|
||||||
|
@trans_write: write transformation
|
||||||
"""
|
"""
|
||||||
dst = self.path_to_dotfile_dst(dst)
|
dst = self.path_to_dotfile_dst(dst)
|
||||||
dotfile = self.get_dotfile_by_src_dst(src, dst)
|
dotfile = self.get_dotfile_by_src_dst(src, dst)
|
||||||
if not dotfile:
|
if not dotfile:
|
||||||
dotfile = self._create_new_dotfile(src, dst, link, chmod=chmod)
|
# add the dotfile
|
||||||
|
dotfile = self._create_new_dotfile(src, dst, link, chmod=chmod,
|
||||||
|
trans_read=trans_read,
|
||||||
|
trans_write=trans_write)
|
||||||
|
|
||||||
if not dotfile:
|
if not dotfile:
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
# add to profile
|
||||||
key = dotfile.key
|
key = dotfile.key
|
||||||
ret = self.cfgyaml.add_dotfile_to_profile(key, self.profile_key)
|
ret = self.cfgyaml.add_dotfile_to_profile(key, self.profile_key)
|
||||||
if ret:
|
if ret:
|
||||||
msg = 'new dotfile {} to profile {}'
|
msg = 'new dotfile {} to profile {}'
|
||||||
self.log.dbg(msg.format(key, self.profile_key))
|
self.log.dbg(msg.format(key, self.profile_key))
|
||||||
|
|
||||||
|
# save the config and reload it
|
||||||
if ret:
|
if ret:
|
||||||
self._save_and_reload()
|
self._save_and_reload()
|
||||||
return ret
|
return ret
|
||||||
@@ -205,15 +213,26 @@ class CfgAggregator:
|
|||||||
# accessors for public methods
|
# accessors for public methods
|
||||||
########################################################
|
########################################################
|
||||||
|
|
||||||
def _create_new_dotfile(self, src, dst, link, chmod=None):
|
def _create_new_dotfile(self, src, dst, link, chmod=None,
|
||||||
|
trans_read=None, trans_write=None):
|
||||||
"""create a new dotfile"""
|
"""create a new dotfile"""
|
||||||
# get a new dotfile with a unique key
|
# get a new dotfile with a unique key
|
||||||
key = self._get_new_dotfile_key(dst)
|
key = self._get_new_dotfile_key(dst)
|
||||||
self.log.dbg('new dotfile key: {}'.format(key))
|
self.log.dbg('new dotfile key: {}'.format(key))
|
||||||
# add the dotfile
|
# add the dotfile
|
||||||
if not self.cfgyaml.add_dotfile(key, src, dst, link, chmod=chmod):
|
trans_r_key = trans_w_key = None
|
||||||
|
if trans_read:
|
||||||
|
trans_r_key = trans_read.key
|
||||||
|
if trans_write:
|
||||||
|
trans_w_key = trans_write.key
|
||||||
|
if not self.cfgyaml.add_dotfile(key, src, dst, link,
|
||||||
|
chmod=chmod,
|
||||||
|
trans_r_key=trans_r_key,
|
||||||
|
trans_w_key=trans_w_key):
|
||||||
return None
|
return None
|
||||||
return Dotfile(key, dst, src)
|
return Dotfile(key, dst, src,
|
||||||
|
trans_r=trans_read,
|
||||||
|
trans_w=trans_write)
|
||||||
|
|
||||||
########################################################
|
########################################################
|
||||||
# parsing
|
# parsing
|
||||||
@@ -282,7 +301,7 @@ class CfgAggregator:
|
|||||||
# patch trans_w/trans_r in dotfiles
|
# patch trans_w/trans_r in dotfiles
|
||||||
self._patch_keys_to_objs(self.dotfiles,
|
self._patch_keys_to_objs(self.dotfiles,
|
||||||
"trans_r",
|
"trans_r",
|
||||||
self._get_trans_w_args(self._get_trans_r),
|
self._get_trans_w_args(self.get_trans_r),
|
||||||
islist=False)
|
islist=False)
|
||||||
self._patch_keys_to_objs(self.dotfiles,
|
self._patch_keys_to_objs(self.dotfiles,
|
||||||
"trans_w",
|
"trans_w",
|
||||||
@@ -453,7 +472,7 @@ class CfgAggregator:
|
|||||||
return trans
|
return trans
|
||||||
return getit
|
return getit
|
||||||
|
|
||||||
def _get_trans_r(self, key):
|
def get_trans_r(self, key):
|
||||||
"""return the trans_r with this key"""
|
"""return the trans_r with this key"""
|
||||||
try:
|
try:
|
||||||
return next(x for x in self.trans_r if x.key == key)
|
return next(x for x in self.trans_r if x.key == key)
|
||||||
|
|||||||
@@ -394,7 +394,8 @@ class CfgYaml:
|
|||||||
self._dirty = True
|
self._dirty = True
|
||||||
return True
|
return True
|
||||||
|
|
||||||
def add_dotfile(self, key, src, dst, link, chmod=None):
|
def add_dotfile(self, key, src, dst, link, chmod=None,
|
||||||
|
trans_r_key=None, trans_w_key=None):
|
||||||
"""add a new dotfile"""
|
"""add a new dotfile"""
|
||||||
if key in self.dotfiles.keys():
|
if key in self.dotfiles.keys():
|
||||||
return False
|
return False
|
||||||
@@ -404,10 +405,15 @@ class CfgYaml:
|
|||||||
self._dbg('new dotfile dst: {}'.format(dst))
|
self._dbg('new dotfile dst: {}'.format(dst))
|
||||||
self._dbg('new dotfile link: {}'.format(link))
|
self._dbg('new dotfile link: {}'.format(link))
|
||||||
self._dbg('new dotfile chmod: {}'.format(chmod))
|
self._dbg('new dotfile chmod: {}'.format(chmod))
|
||||||
|
self._dbg('new dotfile trans_r: {}'.format(trans_r_key))
|
||||||
|
self._dbg('new dotfile trans_w: {}'.format(trans_w_key))
|
||||||
|
|
||||||
|
# create the dotfile dict
|
||||||
df_dict = {
|
df_dict = {
|
||||||
self.key_dotfile_src: src,
|
self.key_dotfile_src: src,
|
||||||
self.key_dotfile_dst: dst,
|
self.key_dotfile_dst: dst,
|
||||||
}
|
}
|
||||||
|
|
||||||
# link
|
# link
|
||||||
dfl = self.settings[self.key_settings_link_dotfile_default]
|
dfl = self.settings[self.key_settings_link_dotfile_default]
|
||||||
if str(link) != dfl:
|
if str(link) != dfl:
|
||||||
@@ -417,6 +423,15 @@ class CfgYaml:
|
|||||||
if chmod:
|
if chmod:
|
||||||
df_dict[self.key_dotfile_chmod] = str(format(chmod, 'o'))
|
df_dict[self.key_dotfile_chmod] = str(format(chmod, 'o'))
|
||||||
|
|
||||||
|
# trans_r/trans_w
|
||||||
|
if trans_r_key:
|
||||||
|
df_dict[self.key_trans_r] = str(trans_r_key)
|
||||||
|
if trans_w_key:
|
||||||
|
df_dict[self.key_trans_w] = str(trans_w_key)
|
||||||
|
|
||||||
|
if self._debug:
|
||||||
|
self._dbg('dotfile dict: {}'.format(df_dict))
|
||||||
|
|
||||||
# add to global dict
|
# add to global dict
|
||||||
self._yaml_dict[self.key_dotfiles][key] = df_dict
|
self._yaml_dict[self.key_dotfiles][key] = df_dict
|
||||||
self._dirty = True
|
self._dirty = True
|
||||||
|
|||||||
@@ -529,6 +529,7 @@ def cmd_importer(opts):
|
|||||||
paths = opts.import_path
|
paths = opts.import_path
|
||||||
importer = Importer(opts.profile, opts.conf,
|
importer = Importer(opts.profile, opts.conf,
|
||||||
opts.dotpath, opts.diff_command,
|
opts.dotpath, opts.diff_command,
|
||||||
|
opts.variables,
|
||||||
dry=opts.dry, safe=opts.safe,
|
dry=opts.dry, safe=opts.safe,
|
||||||
debug=opts.debug,
|
debug=opts.debug,
|
||||||
keepdot=opts.keepdot,
|
keepdot=opts.keepdot,
|
||||||
@@ -538,7 +539,8 @@ def cmd_importer(opts):
|
|||||||
tmpret = importer.import_path(path, import_as=opts.import_as,
|
tmpret = importer.import_path(path, import_as=opts.import_as,
|
||||||
import_link=opts.import_link,
|
import_link=opts.import_link,
|
||||||
import_mode=opts.import_mode,
|
import_mode=opts.import_mode,
|
||||||
import_transw=opts.import_transw)
|
import_transw=opts.import_transw,
|
||||||
|
import_transr=opts.import_transr)
|
||||||
if tmpret < 0:
|
if tmpret < 0:
|
||||||
ret = False
|
ret = False
|
||||||
elif tmpret > 0:
|
elif tmpret > 0:
|
||||||
|
|||||||
@@ -15,19 +15,21 @@ from dotdrop.utils import strip_home, get_default_file_perms, \
|
|||||||
get_unique_tmp_name, removepath
|
get_unique_tmp_name, removepath
|
||||||
from dotdrop.linktypes import LinkTypes
|
from dotdrop.linktypes import LinkTypes
|
||||||
from dotdrop.comparator import Comparator
|
from dotdrop.comparator import Comparator
|
||||||
|
from dotdrop.templategen import Templategen
|
||||||
|
|
||||||
|
|
||||||
class Importer:
|
class Importer:
|
||||||
"""dotfile importer"""
|
"""dotfile importer"""
|
||||||
|
|
||||||
def __init__(self, profile, conf, dotpath, diff_cmd,
|
def __init__(self, profile, conf, dotpath, diff_cmd,
|
||||||
dry=False, safe=True, debug=False,
|
variables, dry=False, safe=True, debug=False,
|
||||||
keepdot=True, ignore=None):
|
keepdot=True, ignore=None):
|
||||||
"""constructor
|
"""constructor
|
||||||
@profile: the selected profile
|
@profile: the selected profile
|
||||||
@conf: configuration manager
|
@conf: configuration manager
|
||||||
@dotpath: dotfiles dotpath
|
@dotpath: dotfiles dotpath
|
||||||
@diff_cmd: diff command to use
|
@diff_cmd: diff command to use
|
||||||
|
@variables: dictionary of variables for the templates
|
||||||
@dry: simulate
|
@dry: simulate
|
||||||
@safe: ask for overwrite if True
|
@safe: ask for overwrite if True
|
||||||
@debug: enable debug
|
@debug: enable debug
|
||||||
@@ -38,19 +40,25 @@ class Importer:
|
|||||||
self.conf = conf
|
self.conf = conf
|
||||||
self.dotpath = dotpath
|
self.dotpath = dotpath
|
||||||
self.diff_cmd = diff_cmd
|
self.diff_cmd = diff_cmd
|
||||||
|
self.variables = variables
|
||||||
self.dry = dry
|
self.dry = dry
|
||||||
self.safe = safe
|
self.safe = safe
|
||||||
self.debug = debug
|
self.debug = debug
|
||||||
self.keepdot = keepdot
|
self.keepdot = keepdot
|
||||||
self.ignore = ignore or []
|
self.ignore = ignore or []
|
||||||
|
|
||||||
|
self.templater = Templategen(variables=self.variables,
|
||||||
|
base=self.dotpath,
|
||||||
|
debug=self.debug)
|
||||||
|
|
||||||
self.umask = get_umask()
|
self.umask = get_umask()
|
||||||
self.log = Logger(debug=self.debug)
|
self.log = Logger(debug=self.debug)
|
||||||
|
|
||||||
def import_path(self, path, import_as=None,
|
def import_path(self, path, import_as=None,
|
||||||
import_link=LinkTypes.NOLINK,
|
import_link=LinkTypes.NOLINK,
|
||||||
import_mode=False,
|
import_mode=False,
|
||||||
import_transw=""):
|
import_transw="",
|
||||||
|
import_transr=""):
|
||||||
"""
|
"""
|
||||||
import a dotfile pointed by path
|
import a dotfile pointed by path
|
||||||
returns:
|
returns:
|
||||||
@@ -66,18 +74,22 @@ class Importer:
|
|||||||
|
|
||||||
# check transw if any
|
# check transw if any
|
||||||
trans_write = None
|
trans_write = None
|
||||||
|
trans_read = None
|
||||||
if import_transw:
|
if import_transw:
|
||||||
trans_write = self.conf.get_trans_w(import_transw)
|
trans_write = self.conf.get_trans_w(import_transw)
|
||||||
|
if import_transr:
|
||||||
|
trans_read = self.conf.get_trans_r(import_transr)
|
||||||
|
|
||||||
return self._import(path, import_as=import_as,
|
return self._import(path, import_as=import_as,
|
||||||
import_link=import_link,
|
import_link=import_link,
|
||||||
import_mode=import_mode,
|
import_mode=import_mode,
|
||||||
trans_write=trans_write)
|
trans_write=trans_write,
|
||||||
|
trans_read=trans_read)
|
||||||
|
|
||||||
def _import(self, path, import_as=None,
|
def _import(self, path, import_as=None,
|
||||||
import_link=LinkTypes.NOLINK,
|
import_link=LinkTypes.NOLINK,
|
||||||
import_mode=False,
|
import_mode=False,
|
||||||
trans_write=None):
|
trans_write=None, trans_read=None):
|
||||||
"""
|
"""
|
||||||
import path
|
import path
|
||||||
returns:
|
returns:
|
||||||
@@ -135,13 +147,14 @@ class Importer:
|
|||||||
if not self._import_file(src, dst, trans_write=trans_write):
|
if not self._import_file(src, dst, trans_write=trans_write):
|
||||||
return -1
|
return -1
|
||||||
|
|
||||||
# TODO add trans_write
|
|
||||||
# TODO add trans_read too
|
|
||||||
return self._import_in_config(path, src, dst, perm, linktype,
|
return self._import_in_config(path, src, dst, perm, linktype,
|
||||||
import_mode)
|
import_mode,
|
||||||
|
trans_w=trans_write,
|
||||||
|
trans_r=trans_read)
|
||||||
|
|
||||||
def _import_in_config(self, path, src, dst, perm,
|
def _import_in_config(self, path, src, dst, perm,
|
||||||
linktype, import_mode):
|
linktype, import_mode,
|
||||||
|
trans_r=None, trans_w=None):
|
||||||
"""
|
"""
|
||||||
import path
|
import path
|
||||||
returns:
|
returns:
|
||||||
@@ -159,7 +172,9 @@ class Importer:
|
|||||||
chmod = perm
|
chmod = perm
|
||||||
|
|
||||||
# add file to config file
|
# add file to config file
|
||||||
retconf = self.conf.new_dotfile(src, dst, linktype, chmod=chmod)
|
retconf = self.conf.new_dotfile(src, dst, linktype, chmod=chmod,
|
||||||
|
trans_read=trans_r,
|
||||||
|
trans_write=trans_w)
|
||||||
if not retconf:
|
if not retconf:
|
||||||
self.log.warn('\"{}\" ignored during import'.format(path))
|
self.log.warn('\"{}\" ignored during import'.format(path))
|
||||||
return 0
|
return 0
|
||||||
@@ -292,7 +307,8 @@ class Importer:
|
|||||||
return path
|
return path
|
||||||
self.log.dbg('executing write transformation {}'.format(trans))
|
self.log.dbg('executing write transformation {}'.format(trans))
|
||||||
tmp = get_unique_tmp_name()
|
tmp = get_unique_tmp_name()
|
||||||
if not trans.transform(path, tmp, debug=self.debug):
|
if not trans.transform(path, tmp, debug=self.debug,
|
||||||
|
templater=self.templater):
|
||||||
msg = 'transformation \"{}\" failed for {}'
|
msg = 'transformation \"{}\" failed for {}'
|
||||||
self.log.err(msg.format(trans.key, path))
|
self.log.err(msg.format(trans.key, path))
|
||||||
if os.path.exists(tmp):
|
if os.path.exists(tmp):
|
||||||
|
|||||||
@@ -60,7 +60,8 @@ Usage:
|
|||||||
dotdrop install [-VbtfndDaW] [-c <path>] [-p <profile>]
|
dotdrop install [-VbtfndDaW] [-c <path>] [-p <profile>]
|
||||||
[-w <nb>] [<key>...]
|
[-w <nb>] [<key>...]
|
||||||
dotdrop import [-Vbdfm] [-c <path>] [-p <profile>] [-i <pattern>...]
|
dotdrop import [-Vbdfm] [-c <path>] [-p <profile>] [-i <pattern>...]
|
||||||
[-l <link>] [-S <key>] [-s <path>] <path>...
|
[--transr=<key>] [--transw=<key>]
|
||||||
|
[-l <link>] [-s <path>] <path>...
|
||||||
dotdrop compare [-LVbz] [-c <path>] [-p <profile>]
|
dotdrop compare [-LVbz] [-c <path>] [-p <profile>]
|
||||||
[-w <nb>] [-C <file>...] [-i <pattern>...]
|
[-w <nb>] [-C <file>...] [-i <pattern>...]
|
||||||
dotdrop update [-VbfdkPz] [-c <path>] [-p <profile>]
|
dotdrop update [-VbfdkPz] [-c <path>] [-p <profile>]
|
||||||
@@ -90,7 +91,8 @@ Options:
|
|||||||
-p --profile=<profile> Specify the profile to use [default: {}].
|
-p --profile=<profile> Specify the profile to use [default: {}].
|
||||||
-P --show-patch Provide a one-liner to manually patch template.
|
-P --show-patch Provide a one-liner to manually patch template.
|
||||||
-s --as=<path> Import as a different path from actual path.
|
-s --as=<path> Import as a different path from actual path.
|
||||||
-S --transw=<key> Apply trans_write key on import.
|
--transr=<key> Associate trans_read key on import.
|
||||||
|
--transw=<key> Apply trans_write key on import.
|
||||||
-t --temp Install to a temporary directory for review.
|
-t --temp Install to a temporary directory for review.
|
||||||
-T --template Only template dotfiles.
|
-T --template Only template dotfiles.
|
||||||
-V --verbose Be verbose.
|
-V --verbose Be verbose.
|
||||||
@@ -275,6 +277,7 @@ class Options(AttrMonitor):
|
|||||||
self.import_ignore.append('*{}'.format(self.install_backup_suffix))
|
self.import_ignore.append('*{}'.format(self.install_backup_suffix))
|
||||||
self.import_ignore = uniq_list(self.import_ignore)
|
self.import_ignore = uniq_list(self.import_ignore)
|
||||||
self.import_transw = self.args['--transw']
|
self.import_transw = self.args['--transw']
|
||||||
|
self.import_transr = self.args['--transr']
|
||||||
|
|
||||||
def _apply_args_update(self):
|
def _apply_args_update(self):
|
||||||
"""update specifics"""
|
"""update specifics"""
|
||||||
|
|||||||
@@ -63,11 +63,13 @@ cfg="${tmps}/config.yaml"
|
|||||||
|
|
||||||
cat > ${cfg} << _EOF
|
cat > ${cfg} << _EOF
|
||||||
trans_read:
|
trans_read:
|
||||||
base64: cat {0} | base64 -d > {1}
|
base64: "cat {0} | base64 -d > {1}"
|
||||||
uncompress: mkdir -p {1} && tar -xf {0} -C {1}
|
decompress: "mkdir -p {1} && tar -xf {0} -C {1}"
|
||||||
|
decrypt: "echo {{@@ profile @@}} | gpg -q --batch --yes --passphrase-fd 0 --no-tty -d {0} > {1}"
|
||||||
trans_write:
|
trans_write:
|
||||||
base64: cat {0} | base64 > {1}
|
base64: "cat {0} | base64 > {1}"
|
||||||
compress: tar -cf {1} -C {0} .
|
compress: "tar -cf {1} -C {0} ."
|
||||||
|
encrypt: "echo {{@@ profile @@}} | gpg -q --batch --yes --passphrase-fd 0 --no-tty -o {1} -c {0}"
|
||||||
config:
|
config:
|
||||||
backup: true
|
backup: true
|
||||||
create: true
|
create: true
|
||||||
@@ -80,25 +82,30 @@ _EOF
|
|||||||
# tokens
|
# tokens
|
||||||
token="test-base64"
|
token="test-base64"
|
||||||
tokend="compressed archive"
|
tokend="compressed archive"
|
||||||
|
tokenenc="encrypted"
|
||||||
|
|
||||||
# create the dotfiles
|
# create the dotfiles
|
||||||
echo ${token} > ${tmpd}/abc
|
echo ${token} > ${tmpd}/abc
|
||||||
mkdir -p ${tmpd}/def/a
|
mkdir -p ${tmpd}/def/a
|
||||||
echo ${tokend} > ${tmpd}/def/a/file
|
echo ${tokend} > ${tmpd}/def/a/file
|
||||||
|
echo ${tokenenc} > ${tmpd}/ghi
|
||||||
|
|
||||||
###########################
|
###########################
|
||||||
# test import
|
# test import
|
||||||
###########################
|
###########################
|
||||||
|
|
||||||
echo "[+] run import"
|
echo "[+] run import"
|
||||||
# import file
|
# import file (to base64)
|
||||||
cd ${ddpath} | ${bin} import -f -c ${cfg} -p p1 -b -V -S base64 ${tmpd}/abc
|
cd ${ddpath} | ${bin} import -f -c ${cfg} -p p1 -b -V --transw=base64 --transr=base64 ${tmpd}/abc
|
||||||
# import directory
|
# import directory (to compress)
|
||||||
cd ${ddpath} | ${bin} import -f -c ${cfg} -p p1 -b -V -S compress ${tmpd}/def
|
cd ${ddpath} | ${bin} import -f -c ${cfg} -p p1 -b -V --transw=compress --transr=decompress ${tmpd}/def
|
||||||
|
# import file (to encrypt)
|
||||||
|
cd ${ddpath} | ${bin} import -f -c ${cfg} -p p1 -b -V --transw=encrypt --transr=decrypt ${tmpd}/ghi
|
||||||
|
|
||||||
# check file imported in dotpath
|
# check file imported in dotpath
|
||||||
[ ! -e ${tmps}/dotfiles/${tmpd}/abc ] && echo "abc does not exist" && exit 1
|
[ ! -e ${tmps}/dotfiles/${tmpd}/abc ] && echo "abc does not exist" && exit 1
|
||||||
[ ! -e ${tmps}/dotfiles/${tmpd}/def ] && echo "def does not exist" && exit 1
|
[ ! -e ${tmps}/dotfiles/${tmpd}/def ] && echo "def does not exist" && exit 1
|
||||||
|
[ ! -e ${tmps}/dotfiles/${tmpd}/ghi ] && echo "ghi does not exist" && exit 1
|
||||||
|
|
||||||
# check content in dotpath
|
# check content in dotpath
|
||||||
echo "checking content"
|
echo "checking content"
|
||||||
@@ -110,17 +117,49 @@ file ${tmps}/dotfiles/${tmpd}/def | grep -i 'tar'
|
|||||||
tar -cf ${tmps}/test-def -C ${tmpd}/def .
|
tar -cf ${tmps}/test-def -C ${tmpd}/def .
|
||||||
diff ${tmps}/dotfiles/${tmpd}/def ${tmps}/test-def
|
diff ${tmps}/dotfiles/${tmpd}/def ${tmps}/test-def
|
||||||
|
|
||||||
|
file ${tmps}/dotfiles/${tmpd}/ghi | grep -i 'gpg symmetrically encrypted data'
|
||||||
|
echo p1 | gpg -q --batch --yes --passphrase-fd 0 --no-tty -d ${tmps}/dotfiles/${tmpd}/ghi > ${tmps}/test-ghi
|
||||||
|
diff ${tmps}/test-ghi ${tmpd}/ghi
|
||||||
|
|
||||||
# check is imported in config
|
# check is imported in config
|
||||||
echo "checking imported in config"
|
echo "checking imported in config"
|
||||||
cd ${ddpath} | ${bin} -p p1 -c ${cfg} files
|
cd ${ddpath} | ${bin} -p p1 -c ${cfg} files
|
||||||
cd ${ddpath} | ${bin} -p p1 -c ${cfg} files | grep '^f_abc'
|
cd ${ddpath} | ${bin} -p p1 -c ${cfg} files | grep '^f_abc'
|
||||||
cd ${ddpath} | ${bin} -p p1 -c ${cfg} files | grep '^d_def'
|
cd ${ddpath} | ${bin} -p p1 -c ${cfg} files | grep '^d_def'
|
||||||
|
cd ${ddpath} | ${bin} -p p1 -c ${cfg} files | grep '^f_ghi'
|
||||||
|
|
||||||
# check has trans_write in config
|
# check has trans_write in config
|
||||||
echo "checking trans_write is set in config"
|
echo "checking trans_write is set in config"
|
||||||
|
echo "--------------"
|
||||||
cat ${cfg}
|
cat ${cfg}
|
||||||
cat ${cfg} | grep -m 1 -A 3 'f_abc' | grep 'trans_write: base64'
|
echo "--------------"
|
||||||
cat ${cfg} | grep -m 1 -A 3 'd_def' | grep 'trans_write: compress'
|
cat ${cfg} | grep -m 1 -A 4 'f_abc' | grep 'trans_write: base64'
|
||||||
|
cat ${cfg} | grep -m 1 -A 4 'd_def' | grep 'trans_write: compress'
|
||||||
|
cat ${cfg} | grep -m 1 -A 4 'f_ghi' | grep 'trans_write: encrypt'
|
||||||
|
|
||||||
|
cat ${cfg} | grep -m 1 -A 4 'f_abc' | grep 'trans_read: base64'
|
||||||
|
cat ${cfg} | grep -m 1 -A 4 'd_def' | grep 'trans_read: decompress'
|
||||||
|
cat ${cfg} | grep -m 1 -A 4 'f_ghi' | grep 'trans_read: decrypt'
|
||||||
|
|
||||||
|
# install these
|
||||||
|
rm ${tmpd}/abc
|
||||||
|
rm -r ${tmpd}/def
|
||||||
|
rm ${tmpd}/ghi
|
||||||
|
|
||||||
|
cd ${ddpath} | ${bin} install -f -c ${cfg} -p p1 -b -V
|
||||||
|
|
||||||
|
# test exist
|
||||||
|
[ ! -e ${tmpd}/abc ] && exit 1
|
||||||
|
[ ! -d ${tmpd}/def/a ] && exit 1
|
||||||
|
[ ! -e ${tmpd}/def/a/file ] && exit 1
|
||||||
|
[ ! -e ${tmpd}/ghi ] && exit 1
|
||||||
|
|
||||||
|
# test content
|
||||||
|
cat ${tmpd}/abc
|
||||||
|
cat ${tmpd}/abc | grep "${token}"
|
||||||
|
cat ${tmpd}/def/a/file
|
||||||
|
cat ${tmpd}/def/a/file | grep "${tokend}"
|
||||||
|
cat ${tmpd}/ghi | grep "${tokenenc}"
|
||||||
|
|
||||||
echo "OK"
|
echo "OK"
|
||||||
exit 0
|
exit 0
|
||||||
|
|||||||
@@ -137,6 +137,8 @@ def _fake_args():
|
|||||||
args['--preserve-mode'] = False
|
args['--preserve-mode'] = False
|
||||||
args['--ignore-missing'] = False
|
args['--ignore-missing'] = False
|
||||||
args['--workdir-clear'] = False
|
args['--workdir-clear'] = False
|
||||||
|
args['--transw'] = ''
|
||||||
|
args['--transr'] = ''
|
||||||
# cmds
|
# cmds
|
||||||
args['profiles'] = False
|
args['profiles'] = False
|
||||||
args['files'] = False
|
args['files'] = False
|
||||||
|
|||||||
Reference in New Issue
Block a user