How do I manage a development journal?

Question

I am working solo on small development projects on my own time, using Git. For each project, I like to have a development journal which is associated with that project and not associated with other projects. The journal is a simple markdown file, journal.md.

Here are my requirements for managing the journal in my workflow:

there must be exactly one managed version of journal.md in the project repository; and
I need to be able to make changes to journal.md from any branch, and keep those changes when checking out different branches via git checkout (i.e., the file must not change as a result of switching branches, and must not be deleted when switching branches).

I am looking for workflows (either a sequence of Git commands for switching branches, or a way of storing journal.md, or something else which enables me to work normally in Git while satisfying the above requirements).

I have investigated the following options.

Keep a separate orphan branch journal; commit the initial journal, and journal updates, to this branch and no other branches. This satisfies the first requirement, but the second requirement is trickier due to the behavior of git checkout.
- Suppose I've finished the day and committed my journal updates to journal; then, when starting the next day, I need to switch from journal to master. In this situation, git checkout will delete journal.md, even though I want to keep the file around and modify it.
- Suppose I have a modified version of journal.md while on master, and I want to commit this modified version to journal. In this situation, git checkout will complain that "local changes will be overwritten".
- I could avoid both of these issues if I could add/commit a single file to journal while on master, without actually checking out master. However, I have not found a way to do this.
Keep a separate repository for the journal file for this particular project--e.g., <project>-journal. This obviously doesn't satisfy the first requirement, although it does satisfy the second. The primary problem with this approach is that I have two separate repositories for a single (small) project. This feels...wrong.

Any suggestions would be appreciated.

Do you actually need to make *changes* to that journal, or only *append* to it? In the latter case, you could think outside the box and manage the journal entries as *commit messages* on a `journal` branch and then *generate* the `journal.md` from output of `git log` for that branch. — Jörg W Mittag, Jul 23 '18 at 13:41
Another idea: rig something up with `git notes`, e.g. keep the journal as a note on the root commit. — Jörg W Mittag, Jul 23 '18 at 13:43
https://stackoverflow.com/questions/7933044/commit-a-file-to-a-different-branch-without-checkout — max630, Jul 23 '18 at 15:15
@GregBurghardt The journal is a stream-of-consciousness notebook. I'm literally writing down what I'm thinking, which helps me organize my thoughts and keep track of what I'm doing moment-to-moment. Eventually I narrow down what I'm thinking and translate it into a task, which I then go and program. — A.M., Jul 23 '18 at 20:33
@Ewan So, these are not commit messages. The entry for a particular day is written before, during, _and_ after all commits have been made. — A.M., Jul 23 '18 at 20:34
This kind of sounds like the "journal" is a place where you are doing some initial design work, writing down "TODOs" and reminders, then, right? — Greg Burghardt, Jul 23 '18 at 20:50
Yeah, that's pretty accurate. Design work is a good way to describe it. I'm doing Kaggle (data science) competitions--so, I'll think through solutions, pros/cons, theoretical considerations; then I'll pick a solution and implement it. Then I'll observe the results, think of improvements/do some research on latest methods, then repeat. Lots of the journal is not about the code, but about possible models, preprocessing, etc. — A.M., Jul 23 '18 at 20:57

score 2 · Answer 1 · answered Jul 23 '18 at 15:51

First, I suggest you reconsider the idea about having one 'journal' for the entire repository, regardless of branch. I do something similar for my larger projects (Release-Notes.md, many people also call this ChangeLog).

Example:

But if you are confident in that choice (one journal regardless of branch) - storing it in its own repository is a fairly viable solution, and gets a little closer to your goal when you combine it with git submodules. Make the journal folder a submodule, and keep all branches up to date with the latest version of that submodule.

score 0 · Answer 2 · answered Apr 24 '23 at 16:42

I’ll suggest a modified version of alternative 1:

Use an orphan branch journal
Keep it in a separate worktree

    git worktree add --detach ../journal
    cd ../journal/
    git switch --orphan journal
    touch journal.md
    git add journal.md
    git commit -m 'Init journal'

Now you can always edit this journal by accessing this worktree, which can stay forever checked out on this branch.

score 0 · Answer 3 · answered Apr 25 '23 at 16:13

Store the "real" journal in the project's parent folder, and frequently copy snapshots into the currently checked out branch. So its history will be recorded with each routine $ git commit -a.

A branch that hasn't been checked out in several weeks will have stale journal history, and that's OK. Eventually it will catch up. And when the feature branch is merged down to main, the aggregate history will all be there.

Assuming "append only" (or append "mostly") record keeping, merge conflicts should be trivial to resolve, if they happen at all.

In emacs I find C-c . (CTRL/C dot) a convenient way of adding standardized datestamps to org-mode markdown files.

For projects "foo" and "baz", here is an example layout. Each of my projects has a Makefile to support common command sequences like $ make lint test or $ make release, and I imagine journaling would get folded into that. Or you could write a brief copy-the-journal shell script that is common across projects.

├── baz
│   ├── Makefile
│   ├── cache
│   │   └── baz-journal.md
│   └── src
├── baz-journal.md
├── foo
│   ├── Makefile
│   ├── cache
│   │   └── foo-journal.md
│   └── src
└── foo-journal.md

How do I manage a development journal?

3 Answers3