mirrors/dotdrop
1
0
Fork 0
mirror of https://github.com/deadc0de6/dotdrop.git synced 2026-02-05 13:48:48 +00:00
Code Activity

Merge pull request #21 from deadc0de6/transformations

Browse Source 
Transformations
This commit is contained in:
deadc0de
2018-02-07 17:19:14 +01:00
committed by GitHub
parent e9980be762 4ff0a0dad5
commit d806005fc8
6 changed files with 239 additions and 106 deletions
Download Patch File Download Diff File Expand all files Collapse all files

261
README.md
View File

@@ -59,18 +59,20 @@ why dotdrop rocks.
* [Installation](#installation)
* [Usage](#usage)
* How to
* [Installing dotfiles](#installing-dotfiles)
* [Diffing your local dotfiles with dotdrop](#diffing-your-local-dotfiles-with-dotdrop)
* [Import new dotfiles](#import-new-dotfiles)
* [List the available profiles](#list-the-available-profiles)
* [List configured dotfiles](#list-configured-dotfiles)
* [Execute an action when deploying a dotfile](#execute-an-action-when-deploying-a-dotfile)
* [All dotfiles for a profile](#all-dotfiles-for-a-profile)
* [Include dotfiles from another profile](#include-dotfiles-from-another-profile)
* [Install dotfiles](#install-dotfiles)
* [Compare dotfiles](#compare-dotfiles)
* [Import dotfiles](#import-dotfiles)
* [List profiles](#list-profiles)
* [List dotfiles](#list-dotfiles)
* [Use actions](#use-actions)
* [Use transformations](#use-transformations)
* [Update dotdrop](#update-dotdrop)
* [Update dotfiles](#update-dotfiles)
* [Store sensitive dotfiles](#store-sensitive-dotfiles)
* [Config](#config)
* [Template](#template)
* [Example](#example)
* [People using dotdrop](#people-using-dotdrop)
@@ -78,11 +80,13 @@ why dotdrop rocks.
# Installation
There's two ways of installing and using dotdrop, either [as a submodule](#as-a-submodule)
to your dotfiles git tree or system-wide [through pypi](#with-pypi).
to your dotfiles git tree or system-wide [with pypi](#with-pypi).
Having dotdrop as a submodule guarantees that anywhere your are cloning your dotfiles git tree
from you'll have dotdrop shipped with it. It is the recommended way.
Dotdrop is also available on aur: https://aur.archlinux.org/packages/dotdrop/
## As a submodule
The following will create a repository for your dotfiles and
@@ -223,55 +227,7 @@ Options:
For easy deployment the default profile used by dotdrop reflects the
hostname of the host on which it runs.
## Config file details
The config file (defaults to *config.yaml*) is a yaml file containing
the following entries:
* **config** entry: contains settings for the deployment
* `backup`: create a backup of the dotfile in case it differs from the
one that will be installed by dotdrop
* `create`: create directory hierarchy when installing dotfiles if
it doesn't exist
* `dotpath`: path to the directory containing the dotfiles to be managed
by dotdrop (absolute path or relative to the config file location)
* **dotfiles** entry: a list of dotfiles
* When `link` is true, dotdrop will create a symlink instead of copying. Template generation (as in [template](#template)) is not supported when `link` is true.
* `actions` contains a list of action keys that need to be defined in the **actions** entry below.
```
<dotfile-key-name>:
dst: <where-this-file-is-deployed>
src: <filename-within-the-dotpath>
# Optional
link: <true|false>
actions:
- <action-key>
```
* **profiles** entry: a list of profiles with the different dotfiles that
need to be managed
* `dotfiles`: the dotfiles associated to this profile
* `include`: include all dotfiles from another profile (optional)
```
<some-name-usually-the-hostname>:
dotfiles:
- <some-dotfile-key-name-defined-above>
- <some-other-dotfile-key-name>
- ...
# Optional
include:
- <some-other-profile>
- ...
```
* **actions** entry: a list of action
```
<action-key>: <command-to-execute>
```
## Installing dotfiles
## Install dotfiles
Simply run
```bash
@@ -281,7 +237,7 @@ $ dotdrop.sh install
Use the `--profile` switch to specify a profile if not using
the host's hostname.
## Diffing your local dotfiles with dotdrop
## Compare dotfiles
Compare local dotfiles with dotdrop's defined ones:
```bash
@@ -291,7 +247,7 @@ $ dotdrop.sh compare
The diffing is done by diff in the backend, one can provide specific
options to diff using the `-o` switch.
## Import new dotfiles
## Import dotfiles
Dotdrop allows to import dotfiles directly from the
filesystem. It will copy the dotfile and update the
@@ -300,10 +256,9 @@ config file automatically.
For example to import `~/.xinitrc`
```bash
$ dotdrop.sh import ~/.xinitrc
```
## List the available profiles
## List profiles
```bash
$ dotdrop.sh list
@@ -313,7 +268,7 @@ Dotdrop allows to choose which profile to use
with the *--profile* switch if you use something
else than the default (the hostname).
## List configured dotfiles
## List dotfiles
The following command lists the different dotfiles
configured for a specific profile:
@@ -332,7 +287,7 @@ f_dunstrc (file: "config/dunst/dunstrc", link: False)
-> ~/.config/dunst/dunstrc
```
## Execute an action when deploying a dotfile
## Use actions
It is sometimes useful to execute some kind of action
when deploying a dotfile. For example let's consider
@@ -364,6 +319,143 @@ Thus when `f_vimrc` is installed, the command
`vim +VundleClean! +VundleInstall +VundleInstall! +qall` will
be executed.
## Use transformations
Transformation actions are used to transform a dotfile before it is
installed. They work like [actions](#use-actions) but are executed before the
dotfile is installed to transform the source.
Transformation commands have two arguments:
* **{0}** will be replaced with the dotfile to process
* **{1}** will be replaced with a temporary file to store the result of the transformation
A typical use-case for transformations is when the dotfile needs to be
stored encrypted.
Here's an example of part of a config file to use gpg encrypted dotfiles:
```
dotfiles:
f_secret:
dst: ~/.secret
src: secret
trans:
- gpg
trans:
gpg: gpg2 -q --for-your-eyes-only --no-tty -d {0} > {1}
```
The above config allows to store the dotfile `~/.secret` encrypted in the *dotfiles*
directory and uses gpg to decrypt it when install is run.
Here's how to deploy the above solution:
* import the clear dotfile (creates the correct entries in the config file)
```
./dotdrop.sh import ~/.secret
```
* encrypt the original dotfile
```
<some-gpg-command> ~/.secret
```
* overwrite the dotfile with the encrypted version
```
cp <encrypted-version-of-secret> dotfiles/secret
```
* edit the config file and add the transformation to the dotfile
* commit and push the changes
## Update dotdrop
If used as a submodule, update it with
```bash
$ git submodule foreach git pull origin master
$ git add dotdrop
$ git commit -m 'update dotdrop'
$ git push
```
Through pypi:
```bash
$ sudo pip3 install dotdrop --upgrade
```
## Update dotfiles
Dotfiles managed by dotdrop can be updated using the `update` command.
There are two cases:
* the dotfile doesn't use [templating](#template): the new version of the dotfile is copied to the
*dotfiles* directory and overwrites the old version. If git is used to version the dotfiles stored
by dotdrop, the git command `diff` can be used to view the changes.
* the dotfile uses [templating](#template): the dotfile must be manually updated, the use of
the dotdrop command `compare` can be helpful to identify the changes to apply to the template.
```
$ dotdrop.sh update ~/.vimrc
```
## Store sensitive dotfiles
Two solutions exist, the first one using an unversioned file (see [Environment variables](#environment-variables))
and the second using transformations (see [Transformations](#use-transformations)).
# Config
The config file (defaults to *config.yaml*) is a yaml file containing
the following entries:
* **config** entry: contains settings for the deployment
* `backup`: create a backup of the dotfile in case it differs from the
one that will be installed by dotdrop
* `create`: create directory hierarchy when installing dotfiles if
it doesn't exist
* `dotpath`: path to the directory containing the dotfiles to be managed
by dotdrop (absolute path or relative to the config file location)
* **dotfiles** entry: a list of dotfiles
* When `link` is true, dotdrop will create a symlink instead of copying. Template generation (as in [template](#template)) is not supported when `link` is true.
* `actions` contains a list of action keys that need to be defined in the **actions** entry below.
* `trans` contains a list of transformation keys that need to be defined in the **trans** entry below.
```
<dotfile-key-name>:
dst: <where-this-file-is-deployed>
src: <filename-within-the-dotpath>
# Optional
link: <true|false>
actions:
- <action-key>
trans:
- <transformation-key>
```
* **profiles** entry: a list of profiles with the different dotfiles that
need to be managed
* `dotfiles`: the dotfiles associated to this profile
* `include`: include all dotfiles from another profile (optional)
```
<some-name-usually-the-hostname>:
dotfiles:
- <some-dotfile-key-name-defined-above>
- <some-other-dotfile-key-name>
- ...
# Optional
include:
- <some-other-profile>
- ...
```
* **actions** entry: a list of action
```
<action-key>: <command-to-execute>
```
* **trans** entry: a list of transformations
```
<trans-key>: <command-to-execute>
```
## All dotfiles for a profile
To use all defined dotfiles for a profile, simply use
@@ -406,37 +498,6 @@ profiles:
```
Here profile *host1* contains all the dotfiles defined for *host2* plus `f_xinitrc`.
## Update dotdrop
If used as a submodule, update it with
```bash
$ git submodule foreach git pull origin master
$ git add dotdrop
$ git commit -m 'update dotdrop'
$ git push
```
Through pypi:
```bash
$ sudo pip3 install dotdrop --upgrade
```
## Update dotfiles
Dotfiles managed by dotdrop can be updated using the `update` command.
There are two cases:
* the dotfile doesn't use [templating](#template): the new version of the dotfile is copied to the
*dotfiles* directory and overwrites the old version. If git is used to version the dotfiles stored
by dotdrop, the git command `diff` can be used to view the changes.
* the dotfile uses [templating](#template): the dotfile must be manually updated, the use of
the dotdrop command `compare` can be helpful to identify the changes to apply to the template.
```
$ dotdrop.sh update ~/.vimrc
```
# Template
Dotdrop leverage the power of [jinja2](http://jinja.pocoo.org/) to handle the
@@ -455,11 +516,10 @@ Note that dotdrop uses different delimiters than
## Available variables
### Profile
* `{{@@ profile @@}}` contains the profile provided to dotdrop.
* `{{@@ env['MY_VAR'] @@}}` contains environment variables (see [Environment variables](#environment-variables))
`{{@@ profile @@}}` contains the profile provided to dotdrop. Below example shows how it is used.
### Environment variables
## Environment variables
It's possible to access environment variables inside the templates. This feature can be used like this:
@@ -481,7 +541,6 @@ pass="verysecurepassword"
```
Of course, this file should not be tracked by git (put it in your `.gitignore`).
Then you can invoke dotdrop with the help of an alias like that:
```
## when using dotdrop as a submodule
@@ -621,7 +680,7 @@ $ git push
```
Otherwise, simply install it from pypi as explained [above](#with-pypi)
and get rid of the submodule:
and get rid of the submodule as shown below:
* move to the dotfiles directory where dotdrop is used as a submodule
```bash

22
dotdrop/action.py
View File

@@ -5,6 +5,7 @@ Represent an action in dotdrop
"""
import subprocess
import os
# local imports
from dotdrop.logger import Logger
@@ -18,11 +19,30 @@ class Action:
self.log = Logger()
def execute(self):
ret = 1
self.log.sub('executing \"%s\"' % (self.action))
try:
subprocess.call(self.action, shell=True)
ret = subprocess.call(self.action, shell=True)
except KeyboardInterrupt:
self.log.warn('action interrupted')
return ret == 0
def transform(self, arg0, arg1):
'''execute transformation with {0} and {1}
where {0} is the file to transform and
{1} is the result file'''
if os.path.exists(arg1):
msg = 'transformation destination exists: %s'
self.log.warn(msg % (arg1))
return False
ret = 1
cmd = self.action.format(arg0, arg1)
self.log.sub('transforming with \"%s\"' % (cmd))
try:
ret = subprocess.call(cmd, shell=True)
except KeyboardInterrupt:
self.log.warn('action interrupted')
return ret == 0
def __str__(self):
return 'key:%s -> \"%s\"' % (self.key, self.action)

23
dotdrop/config.py
View File

@@ -18,6 +18,7 @@ class Cfg:
key_config = 'config'
key_dotfiles = 'dotfiles'
key_actions = 'actions'
key_trans = 'trans'
key_dotpath = 'dotpath'
key_profiles = 'profiles'
key_profiles_dots = 'dotfiles'
@@ -26,6 +27,7 @@ class Cfg:
key_dotfiles_dst = 'dst'
key_dotfiles_link = 'link'
key_dotfiles_actions = 'actions'
key_dotfiles_trans = 'trans'
def __init__(self, cfgpath):
if not os.path.exists(cfgpath):
@@ -41,6 +43,8 @@ class Cfg:
# not linked to content
self.actions = {}
# not linked to content
self.trans = {}
# not linked to content
self.prodots = {}
if not self._load_file():
raise ValueError('config is not valid')
@@ -94,6 +98,12 @@ class Cfg:
for k, v in self.content[self.key_actions].items():
self.actions[k] = Action(k, v)
# parse all transformations
if self.key_trans in self.content:
if self.content[self.key_trans] is not None:
for k, v in self.content[self.key_trans].items():
self.trans[k] = Action(k, v)
# parse the profiles
self.profiles = self.content[self.key_profiles]
if self.profiles is None:
@@ -117,8 +127,17 @@ class Cfg:
entries = v[self.key_dotfiles_actions] if \
self.key_dotfiles_actions in v else []
actions = self._parse_actions(self.actions, entries)
self.dotfiles[k] = Dotfile(k, dst, src,
link=link, actions=actions)
entries = v[self.key_dotfiles_trans] if \
self.key_dotfiles_trans in v else []
trans = self._parse_actions(self.trans, entries)
if len(trans) > 0 and link:
msg = 'transformations disabled for \"%s\"' % (dst)
msg += ' as link is True' % (dst)
self.log.warn(msg)
trans = []
self.dotfiles[k] = Dotfile(k, dst, src, link=link,
actions=actions,
trans=trans)
# assign dotfiles to each profile
for k, v in self.profiles.items():

29
dotdrop/dotdrop.py
View File

@@ -45,6 +45,7 @@ CUR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
LOG = Logger()
HOSTNAME = os.uname()[1]
TILD = '~'
TRANS_SUFFIX = 'trans'
BANNER = """ _ _ _
__| | ___ | |_ __| |_ __ ___ _ __
@@ -101,7 +102,29 @@ def install(opts, conf):
if hasattr(dotfile, 'link') and dotfile.link:
r = inst.link(dotfile.src, dotfile.dst)
else:
r = inst.install(t, opts['profile'], dotfile.src, dotfile.dst)
src = dotfile.src
tmp = None
if dotfile.trans:
tmp = '%s.%s' % (src, TRANS_SUFFIX)
err = False
for trans in dotfile.trans:
s = os.path.join(opts['dotpath'], src)
temp = os.path.join(opts['dotpath'], tmp)
if not trans.transform(s, temp):
msg = 'transformation \"%s\" failed for %s'
LOG.err(msg % (trans.key, dotfile.key))
err = True
break
if err:
if tmp and os.path.exists(tmp):
remove(tmp)
continue
src = tmp
r = inst.install(t, opts['profile'], src, dotfile.dst)
if tmp:
tmp = os.path.join(opts['dotpath'], tmp)
if os.path.exists(tmp):
remove(tmp)
if len(r) > 0 and len(dotfile.actions) > 0:
# execute action
for action in dotfile.actions:
@@ -140,6 +163,10 @@ def compare(opts, conf, tmp, focus=None):
return ret
for dotfile in selected:
if dotfile.trans:
msg = 'ignore %s as it uses transformation(s)'
LOG.log(msg % (dotfile.key))
continue
same, diff = inst.compare(t, tmp, opts['profile'],
dotfile.src, dotfile.dst,
opts=opts['dopts'])

5
dotdrop/dotfile.py
View File

@@ -7,7 +7,8 @@ represents a dotfile in dotdrop
class Dotfile:
def __init__(self, key, dst, src, actions=[], link=False):
def __init__(self, key, dst, src,
actions=[], trans=[], link=False):
# key of dotfile in the config
self.key = key
# where to install this dotfile
@@ -18,6 +19,8 @@ class Dotfile:
self.link = link
# list of actions
self.actions = actions
# list of transformations
self.trans = trans
def __str__(self):
return 'key:%s, src: %s, dst: %s, link: %s' % (self.key, self.src,

5
dotdrop/utils.py
View File

@@ -38,6 +38,11 @@ def get_tmpdir():
return tempfile.mkdtemp(prefix='dotdrop-')
def get_tmpfile():
(fd, path) = tempfile.mkstemp(prefix='dotdrop-')
return path
def remove(path):
''' Remove a file / directory / symlink '''
if not os.path.exists(path):