mirrors/dotdrop
1
0
Fork 0
mirror of https://github.com/deadc0de6/dotdrop.git synced 2026-02-07 03:05:37 +00:00
Code Activity

adding mkdocs

Browse Source
This commit is contained in:
deadc0de6
2020-09-12 14:18:53 +02:00
parent 984295db7e
commit d8ee1bd9df
23 changed files with 2507 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

31
docs/howto/append.md Normal file
View File

@@ -0,0 +1,31 @@
Sometimes it might be useful to be able to append some text to a
file. Dotdrop is able to do that with the help of
[actions](https://github.com/deadc0de6/dotdrop/wiki/usage-actions)
and a temporary file.
Below is a config example to append to a file:
```yaml
dynvariables:
tmpfile: mktemp
variables:
somefile_final: ~/.somefile
dotfiles:
f_somefile:
dst: "{{@@ tmpfile @@}}"
src: somefile
actions:
- strip "{{@@ somefile_final @@}}"
- append "{{@@ tmpfile @@}}" "{{@@ somefile_final @@}}"
actions:
pre:
strip: "sed -i '/^# my pattern$/,$d' {0}"
post:
append: "cat {0} >> {1}; rm -f {0}"
```
During installation, the `strip` action is executed before the installation and strips everything from the pattern `# my pattern` to the end of the file. Then the dotfile `somefile` is installed in a temporary location (here `tmpfile`) and finally the post action `append` will append the content of the `tmpfile` to the final dotfile pointed by `somefile_final`
Obviously the dotfile in the dotpath should start with a unique pattern (here `# my pattern`):
```
# my pattern
this is the end
```

54
docs/howto/create-special-files.md Normal file
View File

@@ -0,0 +1,54 @@
One way for creating symlinks (or any other special files) is to use a combination of
[actions](usage-actions) and a *fake* dotfile.
Let's say for example you have a list of directories you want to link
from under `~/.original` to `~/symlinks`.
```bash
$ tree ~/.original
/home/user/.original
├── dir1
├── dir2
└── dir3
```
First you would store these directories names in a text file in your `<dotpath>/links.txt`
```
dir1
dir2
dir3
```
The config file would contain different elements
* a `dynvariables` that will read the above text file
* a few `variables` for the source and destination
* an action that will create the destination directory and symlink those directories
* a *fake* dotfile (with no `src` and no `dst` values) that will be always installed with the above action
```yaml
dynvariables:
links_list: "cat {{@@ _dotdrop_dotpath @@}}/links.txt | xargs"
...
variables:
links_dst: "{{@@ env['HOME'] @@}}/.symlinks"
links_src: "{{@@ env['HOME'] @@}}/.original"
...
actions:
symlink_them: 'mkdir -p "{1}" && for lnk in {0}; do ln -s "{{@@ links_src @@}}/$lnk" "{1}/$lnk"; done'
...
fake:
src:
dst:
actions:
- symlink_them '{{@@ links_list @@}}' '{{@@ links_dst @@}}'
```
The result would be
```bash
$ tree ~/.symlinks
/home/user/.symlinks
├── dir1 -> /home/user/.original/dir1
├── dir2 -> /home/user/.original/dir2
└── dir3 -> /home/user/.original/dir3
```
For reference, see [issue 243](https://github.com/deadc0de6/dotdrop/issues/243)

26
docs/howto/global-config-files.md Normal file
View File

@@ -0,0 +1,26 @@
# How to manage system wide config files with dotdrop
Dotdrop doesn't allow to handle file rights and permissions (at least not directly). Every operations (`mkdir`, `cp`, `mv`, `ln`, file creation) are executed with the rights of the user calling dotdrop. The rights of the stored dotfile are mirrored on the deployed dotfile (`chmod` like). It works well for local/user dotfiles but doesn't allow to manage global/system config files (`/etc` or `/var` for example) directly.
Using dotdrop with `sudo` to handle local **and** global dotfiles in the same *session* is a bad idea as the resulting files will all have messed up owners.
It is therefore recommended to have two different config files (and thus two different *dotpath*) for handling these two uses cases:
* one `config.yaml` for the local/user dotfiles (with its dedicated *dotpath*)
* another config file for the global/system dotfiles (with its dedicated *dotpath*)
The default config file (`config.yaml`) is used when installing the user dotfiles as usual
```bash
# default config file is config.yaml
$ ./dotdrop.sh import <some-dotfile>
$ ./dotdrop.sh install
...
```
A different config file (for example `global-config.yaml` and its associated *dotpath*) is used when installing/managing global dotfiles and is to be used with `sudo` or directly by the root user
```bash
# specifying explicitly the config file with the --cfg switch
$ sudo ./dotdrop.sh import --cfg=global-config.yaml <some-dotfile>
$ sudo ./dotdrop.sh install --cfg=global-config.yaml
...
```

1
docs/howto/index.md Normal file
View File

@@ -0,0 +1 @@
**TODO**

69
docs/howto/merge-files-when-installing.md Normal file
View File

@@ -0,0 +1,69 @@
Dotdrop allows to merge multiple files into one using the jinja2's `include` directive.
For example let's consider you want to keep your `vimrc` split into multiple parts in dotdrop
* `<dotpath>/vimrc.d/top`: top part of the file
* `<dotpath>/vimrc.d/bottom`: bottom part of the file
And you want dotdrop to merge all those files into `~/.vimrc` whenever you process your vimrc with dotdrop.
First make sure `~/.vimrc` is present in your config file
```yaml
...
dotfiles:
f_vimrc:
dst: ~/.vimrc
src: vimrc
profiles:
hostname:
dotfiles:
- f_vimrc
...
```
Note that the subfiles (`vimrc.d/top` and `vimrc.d/bottom`)
are not known to the config and do not need to be.
Edit the stored vimrc file to include the other files, for example:
```bash
$ cat <dotpath>/vimrc
{%@@ include 'vimrc.d/top' @@%}
filetype on
set t_Co=256
set tw=0
set tabstop=2
set shiftwidth=2
set expandtab
set nocompatible
set nomodeline
syntax on
{%@@ include 'vimrc.d/bottom' @@%}
```
The `include` path parameter needs to be relative to your `dotpath`.
Dotdrop will then automagically include the files into your vimrc when handling `f_vimrc`.
# Merge all files in a directory
To include all files in a directory, a combination of
[dynvariables](config-variables#interpreted-variables)
and [jinja2 directives](http://jinja.pocoo.org/docs/2.10/) have to be used.
Let's say all files in `<dotpath>/toinclude` need to be included into a dotfile.
First define a [dynvariables](config-variables#interpreted-variables)
in the config file which will look for files to include in the above directory:
```yaml
dynvariables:
allfiles: "cd {{@@ _dotdrop_dotpath @@}}; find toinclude/ -type f | xargs"
```
Note that `_dotdrop_dotpath` is part of the built-in variables
(for more see [template variables](templating#template-variables)).
And then use the generated list in the dotfile template:
```
{%@@ for f in allfiles.split() @@%}
{%@@ include f @@%}
{%@@ endfor @@%}
```

71
docs/howto/sensitive-dotfiles.md Normal file
View File

@@ -0,0 +1,71 @@
# Sensitive dotfiles
* [Available solutions](#available-solutions)
* [Store encrypted dotfiles](#store-encrypted-dotfiles)
* [Load passphrase from file](#load-passphrase-from-file)
---
# Available solutions
Two solutions exist, the first one using an unversioned file (see [Environment variables](templating#environment-variables))
and the second using transformations (see [Store encrypted dotfiles](#store-encrypted-dotfiles)).
# Store encrypted dotfiles
Here's an example of part of a config file to use gpg encrypted dotfiles:
```yaml
dotfiles:
f_secret:
dst: ~/.secret
src: secret
trans_read: _gpg
trans_read:
_gpg: gpg2 -q --for-your-eyes-only --no-tty -d {0} > {1}
```
The above config allows to store the dotfile `~/.secret` encrypted in the *dotpath*
directory and uses gpg to decrypt it when `install` is run.
Here's how to deploy above solution:
* import the clear dotfile (what creates the correct entries in the config file)
```bash
$ dotdrop import ~/.secret
```
* encrypt the original dotfile
```bash
$ <some-gpg-command> ~/.secret
```
* overwrite the dotfile with the encrypted version
```bash
$ cp <encrypted-version-of-secret> dotfiles/secret
```
* edit the config file and add the transformation to the dotfile
(as shown in the example above)
* commit and push the changes
# Load passphrase from file
Passphrase is retrieved using a script:
```yaml
variables:
gpg_password: "./get-password.sh"
trans_read:
_gpg: "gpg2 --batch --yes --passphrase-file <({{@@ gpg_password @@}}) -q --for-your-eyes-only --no-tty -d {0} > {1}"
```
Passphrase is stored in a file directly
```yaml
variables:
gpg_password_file: "/tmp/the-password"
trans_read:
_gpg: "gpg2 --batch --yes --passphrase-file <(cat {{@@ gpg_password_file @@}}) -q --for-your-eyes-only --no-tty -d {0} > {1}"
```

216
docs/howto/sharing-content.md Normal file
View File

@@ -0,0 +1,216 @@
There are cases in which two or more dotfiles are very similar. For example,
two files exporting environment variables for two projects built with the same
technology (eg. two node.js web servers deployed on AWS). In these cases it's
nice to share as much code as possible across the dotfiles, by leveraging
templating and merging them in the same dotfile in Dotdrop's `dotpath`. Here
are a few suggestions about how to achieve this.
* [Brute force templating](brute-force-templating)
* [Profile variables](profile-variables)
* [Jinja macros](jinja-macros)
# Brute force templating
The first approach is sheer use of templating and Dotdrop
[template variables](#template-variables). In order to do this, we need to:
1. Create the merged dotfile with an arbitrary name somewhere in `dotpath`.
2. Create two `dotfile` entries in `config.yaml`, both having the merged
dotfile as `src`, but their own `dst`.
Here we have an example:
The merged dotfile in `dotpath` (`dotpath/projects/env`):
```bash
# .env
{%@@ if _dotfile_key == 'server0-env' @@%}
{%@@set aws_db_host = 'super-duper.host' @@%}
{%@@set aws_db_port = 4521 @@%}
{%@@ elif _dotfile_key == 'server1-env' @@%}
{%@@set aws_db_host = 'cheaper.host' @@%}
{%@@set aws_db_host = 9632 @@%}
{%@@ endif @@%}
export DB_HOST='{{@@ aws_db_host @@}}'
export DB_PORT='{{@@ aws_db_port @@}}'
```
Part of Dotdrop `config.yaml` file:
```yaml
# config.yaml
dotfiles:
server0-env:
src: projects/env
dst: ~/projects/server0/.env
server1-env:
src: projects/env
dst: ~/projects/server1/.env
```
Installing the dotfile `server0-env` will create an environment file in
`~/projects/server0/.env` with the following content:
```bash
# .env
export DB_HOST='super-duper.host'
export DB_PORT='4521'
```
# Profile variables
Albeit flexible, the previous method is a bit cumbersome for some use cases.
For example, when the dotfiles belong to different profiles, the cleanest
solution consists in using
[profile variables](config-variables#profile-variables). This is achieved by:
1. Creating the merged dotfile with an arbitrary name somewhere in `dotpath`.
2. Adding some variables in the merged dotfile via templating.
3. Overriding them with different values in each profile via profile variables.
4. Typically, making the dotfile `dst` dynamic, as different profiles need
usually to deploy the dotfiles in different locations.
**NOTE**: This technique does *not* require two different `dotfiles` entry in
`config.yaml`.
An example:
The merged dotfile (`dotpath/projects/env`)
```bash
# .env
export DB_HOST='{{@@ aws_db_host @@}}'
export DB_PORT='{{@@ aws_db_port @@}}'
```
Part of Dotdrop `config.yaml` file:
```yaml
# config.yaml
dotfiles:
env:
src: projects/env
dst: '{{@@ server_path @@}}/.env'
profiles:
server0:
dotfiles:
- env
variables:
aws_db_host: super-duper.host
aws_db_port: 4521
server_path: ~/projects/server0
server1:
dotfiles:
- env
variables:
aws_db_host: cheaper.host
aws_db_port: 9632
server_path: ~/projects/server1
```
With this setup, installing the `server1` profile will create an environment
file in `~/projects/server1/.env` with the following content:
```bash
# .env
export DB_HOST='cheaper.host'
export DB_PORT='9632'
```
# Jinja macros
Even though it has cleaner dotfiles, the profile-variable-based procedure can't
be used in two scenarios: when the dotfiles belong to the same profile, and
when variable values require some complex computations. In both cases the brute
force templating approach can be used, but in the latter one it also makes the
dotfiles bloated with "bookkeeping" logic, thus hard to read.
A solution for this relies in leveraging Jinja macros. This method is a
variation of the brute force templating one, where the merged dotfile is
included from many different dotfiles in `dotpath` via jinja macros, rather
than via many `dotfile` entries with the same `src` attribute. This way, the
merged original dotfiles stays clean as in the profile variables solution,
because computations are in other files.
The steps to achieve this are:
1. Creating the merged dotfile with an arbitrary name somewhere in `dotpath`.
2. Wrapping the whole content of the merged dotfile in a jinja macro, with the
necessary parameters.
3. Calling the macro in each original dotfiles, computing the parameters there.
**NOTE**: The merged dotfile will be empty, as it only contains a jinja macro.
If it needs not to be deployed, the `ignoreempty` entry can be set to
`true` in `config.yaml`.
As usual, an example:
The merged dotfile in `dotpath` (`dotpath/projects/env`):
```bash
{%@@ macro env(db_host, db_port) @@%}
# .env
export DB_HOST='{{@@ db_host @@}}'
export DB_PORT='{{@@ db_port @@}}'
{%@@ endmacro @@%}
```
Server0's environment file (`projects/server0/.env`)
```jinja2
{%@@ from projects/env import env @@%}
{%@@ set keyPieces = _dotfile_key.split('-') @@%}
{%@@ if keyPieces[-1] == 'dbg' @@%}
{%@@ set isLocal = keyPieces[-2] == 'local' @@%}
{%@@ set remote_host = 'super-duper-dbg.host'
if not isLocal
else 'localhost' @@%}
{%@@set aws_db_port = 3333 @@%}
{%@@ elif keyPieces[-1] == 'dev' @@%}
{%@@set aws_db_host = 'localhost' @@%}
{%@@set aws_db_host = 4521 @@%}
{%@@ endif @@%}
{{@@ env(db_host, db_port) @@}}
```
Server1's environment file (`projects/server1/.env`)
```jinja2
{%@@ from projects/env import env @@%}
{{@@ env('average-host.com', 9632) @@}}
```
Part of Dotdrop `config.yaml` file:
```yaml
# config.yaml
dotfiles:
server0-env-remote-dbg:
src: projects/server0/.env
dst: ~/projects/server0/.env.remote.dbg
server0-env-local-dbg:
src: projects/server0/.env
dst: ~/projects/server0/.env.local.dbg
server1-env:
src: projects/server1/.env
dst: ~/projects/server1/.env
```
With this configuration, installing the dotfile `server0-env-local-dbg` will
create an environment file in `~/projects/server0/.env.local.dbg` with the
following content:
```bash
# .env
export DB_HOST='localhost'
export DB_PORT='3333'
```

46
docs/howto/special-chars.md Normal file
View File

@@ -0,0 +1,46 @@
* [Detect encoding](#detect-encoding)
* [Special chars](#special-chars)
* [Re-encode](#re-encode)
---
# Detect encoding
Text file encoding can be identified using for example `file -b <file-path>` or in vim
with `:set fileencoding`
Here's an example of encoding that will fully work with dotdrop:
```bash
$ file -b <some-file>
UTF-8 Unicode text, with escape sequences
```
and another that will mislead the `compare` command and return false/inaccurate results:
```bash
$ file -b <some-file>
ISO-8859 text, with escape sequences
```
# Special chars
## CRLF
The use of dotfiles with DOS/Windows line ending (CRLF, `\r\n`) will result in
the comparison (`compare`) returning a difference while there is none.
This is due to Jinja2 stripping CRLF.
One solution is to use `dos2unix` to re-format the dotfiles before adding them to dotdrop.
See https://github.com/deadc0de6/dotdrop/issues/42.
## Non-unicode chars
Jinja2 is not able to process non-unicode chars (http://jinja.pocoo.org/docs/2.10/api/). This means that dotfiles using non-unicode chars can still be fully managed by dotdrop however when comparing the local file with the one stored in dotdrop, `compare` will return a difference even if there is none.
Either replace the non-unicode chars (see below [Re-encode](re-encode)) or accept the fact the comparison shows a difference while there's none.
See https://github.com/deadc0de6/dotdrop/issues/42.
# Re-encode
To change an existing file's encoding, you can use `recode UTF-8 <filename>` (see [recode](https://linux.die.net/man/1/recode)) or in vim `:set fileencoding=utf-8`.

36
docs/howto/store-compressed-directories.md Normal file
View File

@@ -0,0 +1,36 @@
# Store compressed directories
This is an example on how to use transformations (`trans_read` and `trans_write`) to store
compressed directories and deploy them with dotdrop.
Config file:
```yaml
trans_read:
uncompress: "mkdir -p {1} && tar -xf {0} -C {1}"
trans_write:
compress: "tar -cf {1} -C {0} ."
config:
backup: true
create: true
dotpath: dotfiles
dotfiles:
d_somedir:
dst: ~/.somedir
src: somedir
trans_read: uncompress
trans_write: compress
profiles:
p1:
dotfiles:
- d_somedir
```
The *read* transformation `uncompress` is used to execute below command before deploying the dotfile (where `{0}` is the source and `{1}` the destination)
```
mkdir -p {1} && tar -xf {0} -C {1}
```
And the *write* transformation `compress` is run when updating the dotfile directory by compressing it (where `{0}` is the source and `{1}` the destination)
```
tar -cf {1} -C {0} .
```

102
docs/howto/symlinked-dotfiles.md Normal file
View File

@@ -0,0 +1,102 @@
# Symlink dotfiles
Dotdrop offers two ways to symlink a dotfile through its
config entry `link`
* setting `link: link` for a dotfile will symlink `dst` to `src`
* setting `link: link_children` will, for every direct children of `src`, symlink `dst/<childrenX>` to `src/<childrenX>` (see [Link children](#link-children))
Where `src` is considered as the file stored in your *dotpath* and
`dst` as the file located in your `$HOME`.
Note that if the dotfile is using template directives, it will be symlinked into
`~/.config/dotdrop` instead of directly into your *dotpath*
(see [Templating symlinked dotfiles](#templating-symlinked-dotfiles))
* [Link children](#link-children)
* [Templating symlinked dotfiles](#templating-symlinked-dotfiles)
---
# Link children
This feature can be very useful for dotfiles when you don't want the entire
directory to be symlink but still want to keep a clean config files (with a
limited number of entries).
*Make sure to do a backup of your dotfiles with something like `cp -r <my-important-dotfile>{,.bak}`*
A good example of its use is when managing `~/.vim` with dotdrop.
Here's what it looks like when using `link: link`.
```yaml
config:
dotpath: dotfiles
dotfiles:
vim:
dst: ~/.vim
src: vim
link: link
```
The top directory `~/.vim` is symlinked to the `<dotpath>/vim` location
```bash
$ readlink ~/.vim
~/.dotfiles/vim/
$ ls ~/.dotfiles/vim/
after autoload plugged plugin snippets spell swap vimrc
```
As a result, all files under `~/.vim` will be managed by
dotdrop (including unwanted directories like `spell`, `swap`, etc).
A cleaner solution is to use `link_children` which allows to only symlink
files under the dotfile directory. Let's say only `after`, `plugin`, `snippets`, and `vimrc`
need to be managed in dotdrop. `~/.vim` is imported in dotdrop, cleaned off all unwanted
files/directories and then the `link` entry is set to `link_children` in the config file.
```yaml
config:
dotpath: dotfiles
dotfiles:
vim:
dst: ~/.vim/
src: vim
link: link_children
```
Now all children of the `vim` dotfile's directory in the *dotpath* will be symlinked under `~/.vim/`
without affecting the rest of the local files, keeping the config file clean
and all unwanted files only on the local system.
```bash
$ readlink -f ~/.vim
~/.vim
$ tree -L 1 ~/.vim
~/.vim
├── after -> ~/.dotfiles/vim/after
├── autoload
├── plugged
├── plugin -> ~/.dotfiles/vim/plugin
├── snippets -> ~/.dotfiles/vim/snippets
├── spell
├── swap
└── vimrc -> ~/.dotfiles/vim/vimrc
```
# Templating symlinked dotfiles
For dotfiles not using any templating directives, those are directly linked
to dotdrop's `dotpath` directory (see [Config](config)).
When using templating directives however the dotfiles are first installed into
`workdir` (defaults to *~/.config/dotdrop*, see [Config](config))
and then symlinked there.
This applies to both dotfiles with `link: link` and `link: link_children`.
For example
```bash
# with template
/home/user/.xyz -> /home/user/.config/dotdrop/.xyz
# without template
/home/user/.xyz -> /home/user/dotfiles/xyz
```