diff --git a/README.md b/README.md index 6b25e6a..efa2ae2 100644 --- a/README.md +++ b/README.md @@ -59,18 +59,20 @@ why dotdrop rocks. * [Installation](#installation) * [Usage](#usage) +* How to - * [Installing dotfiles](#installing-dotfiles) - * [Diffing your local dotfiles with dotdrop](#diffing-your-local-dotfiles-with-dotdrop) - * [Import new dotfiles](#import-new-dotfiles) - * [List the available profiles](#list-the-available-profiles) - * [List configured dotfiles](#list-configured-dotfiles) - * [Execute an action when deploying a dotfile](#execute-an-action-when-deploying-a-dotfile) - * [All dotfiles for a profile](#all-dotfiles-for-a-profile) - * [Include dotfiles from another profile](#include-dotfiles-from-another-profile) + * [Install dotfiles](#install-dotfiles) + * [Compare dotfiles](#compare-dotfiles) + * [Import dotfiles](#import-dotfiles) + * [List profiles](#list-profiles) + * [List dotfiles](#list-dotfiles) + * [Use actions](#use-actions) + * [Use transformations](#use-transformations) * [Update dotdrop](#update-dotdrop) * [Update dotfiles](#update-dotfiles) + * [Store sensitive dotfiles](#store-sensitive-dotfiles) +* [Config](#config) * [Template](#template) * [Example](#example) * [People using dotdrop](#people-using-dotdrop) @@ -78,11 +80,13 @@ why dotdrop rocks. # Installation There's two ways of installing and using dotdrop, either [as a submodule](#as-a-submodule) -to your dotfiles git tree or system-wide [through pypi](#with-pypi). +to your dotfiles git tree or system-wide [with pypi](#with-pypi). Having dotdrop as a submodule guarantees that anywhere your are cloning your dotfiles git tree from you'll have dotdrop shipped with it. It is the recommended way. +Dotdrop is also available on aur: https://aur.archlinux.org/packages/dotdrop/ + ## As a submodule The following will create a repository for your dotfiles and @@ -223,55 +227,7 @@ Options: For easy deployment the default profile used by dotdrop reflects the hostname of the host on which it runs. -## Config file details - -The config file (defaults to *config.yaml*) is a yaml file containing -the following entries: - -* **config** entry: contains settings for the deployment - * `backup`: create a backup of the dotfile in case it differs from the - one that will be installed by dotdrop - * `create`: create directory hierarchy when installing dotfiles if - it doesn't exist - * `dotpath`: path to the directory containing the dotfiles to be managed - by dotdrop (absolute path or relative to the config file location) - -* **dotfiles** entry: a list of dotfiles - * When `link` is true, dotdrop will create a symlink instead of copying. Template generation (as in [template](#template)) is not supported when `link` is true. - * `actions` contains a list of action keys that need to be defined in the **actions** entry below. -``` - : - dst: - src: - # Optional - link: - actions: - - -``` - -* **profiles** entry: a list of profiles with the different dotfiles that - need to be managed - * `dotfiles`: the dotfiles associated to this profile - * `include`: include all dotfiles from another profile (optional) - -``` - : - dotfiles: - - - - - - ... - # Optional - include: - - - - ... -``` - -* **actions** entry: a list of action -``` - : -``` - -## Installing dotfiles +## Install dotfiles Simply run ```bash @@ -281,7 +237,7 @@ $ dotdrop.sh install Use the `--profile` switch to specify a profile if not using the host's hostname. -## Diffing your local dotfiles with dotdrop +## Compare dotfiles Compare local dotfiles with dotdrop's defined ones: ```bash @@ -291,7 +247,7 @@ $ dotdrop.sh compare The diffing is done by diff in the backend, one can provide specific options to diff using the `-o` switch. -## Import new dotfiles +## Import dotfiles Dotdrop allows to import dotfiles directly from the filesystem. It will copy the dotfile and update the @@ -300,10 +256,9 @@ config file automatically. For example to import `~/.xinitrc` ```bash $ dotdrop.sh import ~/.xinitrc - ``` -## List the available profiles +## List profiles ```bash $ dotdrop.sh list @@ -313,7 +268,7 @@ Dotdrop allows to choose which profile to use with the *--profile* switch if you use something else than the default (the hostname). -## List configured dotfiles +## List dotfiles The following command lists the different dotfiles configured for a specific profile: @@ -332,7 +287,7 @@ f_dunstrc (file: "config/dunst/dunstrc", link: False) -> ~/.config/dunst/dunstrc ``` -## Execute an action when deploying a dotfile +## Use actions It is sometimes useful to execute some kind of action when deploying a dotfile. For example let's consider @@ -364,6 +319,143 @@ Thus when `f_vimrc` is installed, the command `vim +VundleClean! +VundleInstall +VundleInstall! +qall` will be executed. +## Use transformations + +Transformation actions are used to transform a dotfile before it is +installed. They work like [actions](#use-actions) but are executed before the +dotfile is installed to transform the source. + +Transformation commands have two arguments: + +* **{0}** will be replaced with the dotfile to process +* **{1}** will be replaced with a temporary file to store the result of the transformation + +A typical use-case for transformations is when the dotfile needs to be +stored encrypted. + +Here's an example of part of a config file to use gpg encrypted dotfiles: +``` +dotfiles: + f_secret: + dst: ~/.secret + src: secret + trans: + - gpg +trans: + gpg: gpg2 -q --for-your-eyes-only --no-tty -d {0} > {1} +``` + +The above config allows to store the dotfile `~/.secret` encrypted in the *dotfiles* +directory and uses gpg to decrypt it when install is run. + +Here's how to deploy the above solution: + +* import the clear dotfile (creates the correct entries in the config file) +``` +./dotdrop.sh import ~/.secret +``` +* encrypt the original dotfile +``` + ~/.secret +``` +* overwrite the dotfile with the encrypted version +``` +cp dotfiles/secret +``` +* edit the config file and add the transformation to the dotfile +* commit and push the changes + +## Update dotdrop + +If used as a submodule, update it with +```bash +$ git submodule foreach git pull origin master +$ git add dotdrop +$ git commit -m 'update dotdrop' +$ git push +``` + +Through pypi: +```bash +$ sudo pip3 install dotdrop --upgrade +``` + +## Update dotfiles + +Dotfiles managed by dotdrop can be updated using the `update` command. +There are two cases: + + * the dotfile doesn't use [templating](#template): the new version of the dotfile is copied to the + *dotfiles* directory and overwrites the old version. If git is used to version the dotfiles stored + by dotdrop, the git command `diff` can be used to view the changes. + * the dotfile uses [templating](#template): the dotfile must be manually updated, the use of + the dotdrop command `compare` can be helpful to identify the changes to apply to the template. + +``` +$ dotdrop.sh update ~/.vimrc +``` + +## Store sensitive dotfiles + +Two solutions exist, the first one using an unversioned file (see [Environment variables](#environment-variables)) +and the second using transformations (see [Transformations](#use-transformations)). + +# Config + +The config file (defaults to *config.yaml*) is a yaml file containing +the following entries: + +* **config** entry: contains settings for the deployment + * `backup`: create a backup of the dotfile in case it differs from the + one that will be installed by dotdrop + * `create`: create directory hierarchy when installing dotfiles if + it doesn't exist + * `dotpath`: path to the directory containing the dotfiles to be managed + by dotdrop (absolute path or relative to the config file location) + +* **dotfiles** entry: a list of dotfiles + * When `link` is true, dotdrop will create a symlink instead of copying. Template generation (as in [template](#template)) is not supported when `link` is true. + * `actions` contains a list of action keys that need to be defined in the **actions** entry below. + * `trans` contains a list of transformation keys that need to be defined in the **trans** entry below. +``` + : + dst: + src: + # Optional + link: + actions: + - + trans: + - +``` + +* **profiles** entry: a list of profiles with the different dotfiles that + need to be managed + * `dotfiles`: the dotfiles associated to this profile + * `include`: include all dotfiles from another profile (optional) + +``` + : + dotfiles: + - + - + - ... + # Optional + include: + - + - ... +``` + +* **actions** entry: a list of action +``` + : +``` + +* **trans** entry: a list of transformations +``` + : +``` + ## All dotfiles for a profile To use all defined dotfiles for a profile, simply use @@ -406,37 +498,6 @@ profiles: ``` Here profile *host1* contains all the dotfiles defined for *host2* plus `f_xinitrc`. -## Update dotdrop - -If used as a submodule, update it with -```bash -$ git submodule foreach git pull origin master -$ git add dotdrop -$ git commit -m 'update dotdrop' -$ git push -``` - -Through pypi: -```bash -$ sudo pip3 install dotdrop --upgrade -``` - - -## Update dotfiles - -Dotfiles managed by dotdrop can be updated using the `update` command. -There are two cases: - - * the dotfile doesn't use [templating](#template): the new version of the dotfile is copied to the - *dotfiles* directory and overwrites the old version. If git is used to version the dotfiles stored - by dotdrop, the git command `diff` can be used to view the changes. - * the dotfile uses [templating](#template): the dotfile must be manually updated, the use of - the dotdrop command `compare` can be helpful to identify the changes to apply to the template. - -``` -$ dotdrop.sh update ~/.vimrc -``` - # Template Dotdrop leverage the power of [jinja2](http://jinja.pocoo.org/) to handle the @@ -455,11 +516,10 @@ Note that dotdrop uses different delimiters than ## Available variables -### Profile +* `{{@@ profile @@}}` contains the profile provided to dotdrop. +* `{{@@ env['MY_VAR'] @@}}` contains environment variables (see [Environment variables](#environment-variables)) -`{{@@ profile @@}}` contains the profile provided to dotdrop. Below example shows how it is used. - -### Environment variables +## Environment variables It's possible to access environment variables inside the templates. This feature can be used like this: @@ -481,7 +541,6 @@ pass="verysecurepassword" ``` Of course, this file should not be tracked by git (put it in your `.gitignore`). - Then you can invoke dotdrop with the help of an alias like that: ``` ## when using dotdrop as a submodule @@ -621,7 +680,7 @@ $ git push ``` Otherwise, simply install it from pypi as explained [above](#with-pypi) -and get rid of the submodule: +and get rid of the submodule as shown below: * move to the dotfiles directory where dotdrop is used as a submodule ```bash diff --git a/dotdrop/action.py b/dotdrop/action.py index e463f4d..387cc1c 100644 --- a/dotdrop/action.py +++ b/dotdrop/action.py @@ -5,6 +5,7 @@ Represent an action in dotdrop """ import subprocess +import os # local imports from dotdrop.logger import Logger @@ -18,11 +19,30 @@ class Action: self.log = Logger() def execute(self): + ret = 1 self.log.sub('executing \"%s\"' % (self.action)) try: - subprocess.call(self.action, shell=True) + ret = subprocess.call(self.action, shell=True) except KeyboardInterrupt: self.log.warn('action interrupted') + return ret == 0 + + def transform(self, arg0, arg1): + '''execute transformation with {0} and {1} + where {0} is the file to transform and + {1} is the result file''' + if os.path.exists(arg1): + msg = 'transformation destination exists: %s' + self.log.warn(msg % (arg1)) + return False + ret = 1 + cmd = self.action.format(arg0, arg1) + self.log.sub('transforming with \"%s\"' % (cmd)) + try: + ret = subprocess.call(cmd, shell=True) + except KeyboardInterrupt: + self.log.warn('action interrupted') + return ret == 0 def __str__(self): return 'key:%s -> \"%s\"' % (self.key, self.action) diff --git a/dotdrop/config.py b/dotdrop/config.py index ff0ff04..29a5c20 100644 --- a/dotdrop/config.py +++ b/dotdrop/config.py @@ -18,6 +18,7 @@ class Cfg: key_config = 'config' key_dotfiles = 'dotfiles' key_actions = 'actions' + key_trans = 'trans' key_dotpath = 'dotpath' key_profiles = 'profiles' key_profiles_dots = 'dotfiles' @@ -26,6 +27,7 @@ class Cfg: key_dotfiles_dst = 'dst' key_dotfiles_link = 'link' key_dotfiles_actions = 'actions' + key_dotfiles_trans = 'trans' def __init__(self, cfgpath): if not os.path.exists(cfgpath): @@ -41,6 +43,8 @@ class Cfg: # not linked to content self.actions = {} # not linked to content + self.trans = {} + # not linked to content self.prodots = {} if not self._load_file(): raise ValueError('config is not valid') @@ -94,6 +98,12 @@ class Cfg: for k, v in self.content[self.key_actions].items(): self.actions[k] = Action(k, v) + # parse all transformations + if self.key_trans in self.content: + if self.content[self.key_trans] is not None: + for k, v in self.content[self.key_trans].items(): + self.trans[k] = Action(k, v) + # parse the profiles self.profiles = self.content[self.key_profiles] if self.profiles is None: @@ -117,8 +127,17 @@ class Cfg: entries = v[self.key_dotfiles_actions] if \ self.key_dotfiles_actions in v else [] actions = self._parse_actions(self.actions, entries) - self.dotfiles[k] = Dotfile(k, dst, src, - link=link, actions=actions) + entries = v[self.key_dotfiles_trans] if \ + self.key_dotfiles_trans in v else [] + trans = self._parse_actions(self.trans, entries) + if len(trans) > 0 and link: + msg = 'transformations disabled for \"%s\"' % (dst) + msg += ' as link is True' % (dst) + self.log.warn(msg) + trans = [] + self.dotfiles[k] = Dotfile(k, dst, src, link=link, + actions=actions, + trans=trans) # assign dotfiles to each profile for k, v in self.profiles.items(): diff --git a/dotdrop/dotdrop.py b/dotdrop/dotdrop.py index c4b2748..4ef9c1a 100644 --- a/dotdrop/dotdrop.py +++ b/dotdrop/dotdrop.py @@ -45,6 +45,7 @@ CUR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) LOG = Logger() HOSTNAME = os.uname()[1] TILD = '~' +TRANS_SUFFIX = 'trans' BANNER = """ _ _ _ __| | ___ | |_ __| |_ __ ___ _ __ @@ -101,7 +102,29 @@ def install(opts, conf): if hasattr(dotfile, 'link') and dotfile.link: r = inst.link(dotfile.src, dotfile.dst) else: - r = inst.install(t, opts['profile'], dotfile.src, dotfile.dst) + src = dotfile.src + tmp = None + if dotfile.trans: + tmp = '%s.%s' % (src, TRANS_SUFFIX) + err = False + for trans in dotfile.trans: + s = os.path.join(opts['dotpath'], src) + temp = os.path.join(opts['dotpath'], tmp) + if not trans.transform(s, temp): + msg = 'transformation \"%s\" failed for %s' + LOG.err(msg % (trans.key, dotfile.key)) + err = True + break + if err: + if tmp and os.path.exists(tmp): + remove(tmp) + continue + src = tmp + r = inst.install(t, opts['profile'], src, dotfile.dst) + if tmp: + tmp = os.path.join(opts['dotpath'], tmp) + if os.path.exists(tmp): + remove(tmp) if len(r) > 0 and len(dotfile.actions) > 0: # execute action for action in dotfile.actions: @@ -140,6 +163,10 @@ def compare(opts, conf, tmp, focus=None): return ret for dotfile in selected: + if dotfile.trans: + msg = 'ignore %s as it uses transformation(s)' + LOG.log(msg % (dotfile.key)) + continue same, diff = inst.compare(t, tmp, opts['profile'], dotfile.src, dotfile.dst, opts=opts['dopts']) diff --git a/dotdrop/dotfile.py b/dotdrop/dotfile.py index 3a3c75b..657385b 100644 --- a/dotdrop/dotfile.py +++ b/dotdrop/dotfile.py @@ -7,7 +7,8 @@ represents a dotfile in dotdrop class Dotfile: - def __init__(self, key, dst, src, actions=[], link=False): + def __init__(self, key, dst, src, + actions=[], trans=[], link=False): # key of dotfile in the config self.key = key # where to install this dotfile @@ -18,6 +19,8 @@ class Dotfile: self.link = link # list of actions self.actions = actions + # list of transformations + self.trans = trans def __str__(self): return 'key:%s, src: %s, dst: %s, link: %s' % (self.key, self.src, diff --git a/dotdrop/utils.py b/dotdrop/utils.py index 45765d5..c07eada 100644 --- a/dotdrop/utils.py +++ b/dotdrop/utils.py @@ -38,6 +38,11 @@ def get_tmpdir(): return tempfile.mkdtemp(prefix='dotdrop-') +def get_tmpfile(): + (fd, path) = tempfile.mkstemp(prefix='dotdrop-') + return path + + def remove(path): ''' Remove a file / directory / symlink ''' if not os.path.exists(path):