Posts on My mostly tech blog

Are all mouses defective by design?

Tue, 15 Apr 2025 00:00:00 -0400

How it all started

It all started 2 years ago when I had to replace a broken 10yo mouse I had been using extensively.

I bought another Logitech mouse, but it started having weird double click issues at random time. It would be less of an issue if I pressed the button harder and at the center.

I thought this was a fluke and ordered another mouse, to get the same issue shortly after. The problem seems to appear between 6 months to 1 year with my current daily use of the mouse.

One could claim I abuse the mouse too much with 9 hours of work and more browsing / gaming after work, but I had another mouse for 10 years without a single issue.

What to do next?

I started searching the internet for people with similar issue and solutions.

I found plenty (>20 posts with similar information) of problem with the exact issue.

What is the actual problem

The root cause is coming from the two facts:

micro switches in mouse were always operated our of their electrical spec. you must call it within certain volts or it will not have the listed longevity
with the breakthrough of wireless mouse, circuits were re-designed to optimize operating in much lower voltage than when we only had wired mouse

With older wired mouse, it looks like even though the voltage is lower than expected, it’s still not bad and will operate relatively close to spec. So the longevity is not altered significantly.

With wireless mouse, this is way too under voltage (1.3 ish volt vs 5 ish volts), and creates oxidation on the micro switch contacts.

You can watch a Alex Denis (a geek) analyzing this for hours here:

Solutions

Solution 1 - Blowing warm humid air into the mouse

Many “fixed” their issue by regularly blowing into the mouse buttons with their warm breath.

I tried that, and oh yeah, it fixed it.

The problem is it keeps coming back, after couple of minutes up to 1-2 hours.

Solution 2 - Switch to a wired mouse

Does not work, circuits within modern wired mouse operate like wireless mouse for cost optimization.

Solution 3 - Open the mouse, the switches and bend the actuator

Does not work permanently, this only fixes the issue for a bit of time

Solution 4 - Open the mouse and replace all switches

I tried that, but it worked only for a time.

Solution 5 - Find better mouse that use switches less prone to this issue

Plausible solution, but finding the right mouse is a bit challenging.

Solution 6 - Use a mouse with optical switches

I ended up picking this solution. I found myself a mouse that uses optical switches that is not subject to the double click bug.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Back to Doom Emacs

Sun, 13 Apr 2025 18:19:59 -0400

In the beginning

It all started last year (was it before last year?) where I decided to simplify my overall tech setup, and put more Clojure (Joy) in my tech life.

Some steps to simplify my life:

Migrate my personal laptop from Guix back to Nix
Rebuild my servers from my half baked docker swarm.
Going back to Emacs native bindings

Now is time for a 2nd round of Emacs changes.

My current Emacs usage

In recent time, my use of Emacs is as followed:

IRC with ERC, to chat with my Geek friends
Email with Notmuch
Notes with Denote
Clojure coding with Cider

My current challenges with this my from scratch setup

Here are some of the challenges I have:

Taking notes
- Completion does not always work in my notes
- I would like to link to sub sections of org mode, which it works, completion does not and I would have to code something for it
IRC
- I always have issues connecting with ERC, the support for authinfo kind of work, but on restart, I always have to connect, disconnect, and connect again, and C-g to abort the password prompt.
- lots of connect/disconnect, which I want to track, but not for people that don’t talk.
- my custom layout is kind of broken, not aligning properly
Email
- While I like the Notmuch approach, I found out I don’t really use labels and the overall setup to delete or move elements in actual folders is complex and only partially working
- I’m thinking of trying again mu4e, which has a more traditional approach to email, but I don’t have a setup yet
Clojure
- I broke my LSP setup and I used LSP to get better completion
- I’d like to get better Java integration in my IDE since Clojure integrates with Java
Completion
- I want to try Corfu, but doing so I broke my completion in couple of places.
- I tried putting back company, but ended up with an incomplete setup

Why not my from scratch setup?

Some of the key reasons:

Even though my from scratch setup is nice, I still have some integration issues with completion at point in some scenarios, and sometimes completion stop working and I don’t know why
I always get it close to right, but not exactly right.
I’m tired of setting up the basics on Emacs
My setup is getting slower
I broke my LSP setup and I’m tired of fixing it
It was a good learning, but want to move on and do other things
Simplify my tech life, I realized I prefer to code rather than configure my IDE
There are more Emacs packages I want to use to simplify my tech life, and I don’t want to duplicate effort if another Emacs distribution has it.

Why Doom?

Doom ticks all the boxes of what I need, what I want to improve on my setup, while keeping my life simple.

Yes it’s not fully under my control, and it’s someone else’s setup, but:

why should that be different than using a Linux distro, and the fact that it’s someone else’s Linux? No one plan to roll their own Linux (well almost no one :D)
it’s that or I will stop using Emacs gradually
It’s core is solid and maintained by many people
It’s core works the way I need it
The Doom configuration modules cover 80% of what I use with no to minimal configuration
Even though it favors vim bindings, It supports Emacs native bindings well
I can easily extend it, and I already have, with the advantage of potentially contributing to the community
It’s fast to load
Issues are fixed by the community which I also can contribute to

What is next

Some of my next Emacs objectives are to improve my note taking, and Email setup.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Vacation Affects My Blogging

Sun, 08 Sep 2024 00:00:00 -0400

Today I realized I did not think about posting since I’m back from vacation. Whenever I’m on vacation, I think about this blog. When I work, well I forget about it.

My resolution is to post every week, no matter the length or the topic.

And looks like I skipped one!

Lets try to fix that.

what happened since my last post?

back to work

Lots of things to deal with, but nothing really interesting. like most back to work, it’s busy.

Parents house getting flooded and having to deal with that.

Work is typical, flooding not so much.

The whole area got lot more rain than usual, and lot more than drain systems could handle.

Paired with water, power outage that made the pumps stop.

We have another month or two to deal with all of this:

insurance, they are busy, and super hard to reach
some shady but popular disaster management company abusing (we fired them)
we finally had really good contractors to remove drywall, floors, and kill mold that was forming already.
we still have to finalize the total cost of the loss, reconstruction and submit claims to the insurance.
and get the job done…

installing a new car radio

A eonon X3 Linux based car radio.

Got to learn how to install backup cam, and remove lots of inside trims of a Toyota corolla.

I’m not done yet, I should post later about that project when it’s completed.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Rebuilding My Server - Proxmox Cloud Init Image template

Wed, 28 Aug 2024 00:00:00 -0400

Cloud init

According to cloud-init.io, Cloud init is:

Cloud images are operating system templates and every instance starts out as an identical clone of every other instance. It is the user data that gives every cloud instance its personality and cloud-init is the tool that applies user data to your instances automatically.

Okay…

Maybe if I explain how I create a VM from a cloud-init image:

Clone the cloud-init Ubuntu server template
In Proxmox UI, supply key information like name, IP/network
Enlarge the main disk and tweak resources allocated
Start the VM

No lengthy installation from ISO, no additional OS specific GUI. All cloud-init capable OS are configured the same way.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Time for Some Changes to My Primary Server

Tue, 27 Aug 2024 18:00:00 -0400

My selfhosting Journey

My journey to self hosting started in the 90s and evolved (good and bad) over the past 25 or so years:

Created a Linux router using a 486, some ISA BNC network cards, and a dialup connection with a remote dialup controller.
Self hosting email
HTPC
Self hosting Plex, Kodi, RSS, Music collection, pictures, books, Nextcloud and more
home made NAS
Freenas (The old TrueNAS)
Openmediavault
Truenas Core
Truenas Scale
Docker servers
Docker swarm orchestration
Proxmox
XCP-NG / Xen Orchestra

My current setup is:

TrueNAS Scale
XCP-NG / Xen Orchestra
3 Docker swarm clusters (private, public, infra)
48 ports layer 3 swich, 2 access points
VLAN to separate my networks (dmz, private, infra, untrusted)

VM platform

User of both Proxmox and XCP-NG / Xen Orchestra for couple of years.

Running XCP-NG / Xen orchestra (XOA) at home means:

XCP-NG is rock solid
XCP-NG does not come with a web management console and the TUI console is quite limited
you have two separate components to manage (XCP-NG nodes and XOA), but you can re-use XOA for multiple XCP-NG nodes.
You can migrate VMs from one node to another without downtime. I tried this once, but never used since.
The community edition of XOA has some limitations
You can get Complete edition of XOA by spawning a VM and compiling it using the awesome XenOrchestraInstallerUpdater script
You need to manually upgrade Complete edition of XOA
You get a bit more complex cloud init config, but at the benefit of allowing to supply any of the user, meta, provider cloud init files directly in the web UI
Dashboard is minimal.
KVM hypervisor

Running Proxmox at home means:

One component to manage
automated updates
LXC containers
Nicer default Dashboard
Simple cloud init configuration, but more limited. You need to do some CLI do extend some of the files.
Simpler to manage
As far as I remember, seamless migration from one node to another is not supported
Has more complete off the shelf features
KVM hypervisor
A lot more YouTube tutorials for Proxmox
quite good CLI

My main decider for going back to Proxmox: automated updates, a better web console and I don’t need seamless VM migration.

Otherwise they are quite similar.

Container orchestration platform

I’m running two docker swarm clusters to distribute my apps. It’s working OK, but Docker swarm seems to be quite dead (or stable).

The other main Container Orchestration platform is Kubernetes.

Why I’m switching to Kubernetes:

Active community
Many lightweight Kubernetes platform, like K0S, K3S, Minikube and more.
I use it at work as a developer
I want to learn more of it’s components and administration
I also want to learn Helm, the Kubernetes “package manager”

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Rebuilding My Server Getting Unsupported Sfp+ Working on Linux

Tue, 27 Aug 2024 00:00:00 -0400

Context

I have been running VMs on XCP-NG for a while now. XCP-NG does not come with a console by default, but Xen Orchestra (XOA) gets you covered. The only issue with XOA is the complete version requires you to compile and install it manually.

I’ve been lagging on this setup and remembered a time where I ran Proxmox.

So 2 days ago, I installed a HD in one of my kid’s desktop (don’t tell them), and moved my most critical services: plex and miniflux.

Today, I installed Proxmox.

Network Challenges

My server has 7 network ports:

one IPMI admin port locked on a restricted network
4 onboard gigabit ports
2 SFP+ 10GB modules installed in a X520 PCIe network card

With that many ports, it could be challenge to find the proper port to setup in the network bridge.

My first issue? I got lost with so many ports and picked the wrong port.

My second issue? I did not notice my X520 card was not being detected properly.

The ixgbe kernel module needed for the X520 card reports the SPF+ modules as “Unsupported”.

There is a trick

You can tell ixgbe module to support the “unsupported” SPF+ modules.

With grub in /etc/default/grub

GRUB_CMDLINE_LINUX=”ixgbe.allow_unsupported_sfp=1″

For two modules:

GRUB_CMDLINE_LINUX=”ixgbe.allow_unsupported_sfp=1,1″

run update-grub and reboot.

When typing dmesg | grep ixgbe I now see the modules being detected.

Note to future self: type sfp instead spf to save some time.

Alternative

The same can be achieved by setting a module configuration:

`1`	`echo "options ixgbe allow_unsupported_sfp=1,1" > /etc/modprobe.d/ixgbe.conf`

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Owning Your Own Domain(s)

Sun, 18 Aug 2024 16:59:12 -0400

Here is how I use my domains.

I have two domains:

One for my email addresses
One for my own network and servers

I use two different ones to separate usage, and to keep one semi private.

Benefits

Some benefits:

I’m not locked in a particular service provider For example, I used many email providers, yet, kept the same email address. Been with mailbox.org, mxroute.com, migadu, and now I’m on fastmail.
My own personalized email address
People make me confirm my email address constantly :-D
Able to do dynamic DNS with your own domain to simplify remote connections
I use a real domain for my local network, so things works regardless if I’m on WiFi or remote.

DNS Providers

I’m using namecheap, but heard good things from gandi.

Should you get your own?

Yes if you want your own email domain and move away from gmail / hotmail services. Yes if you want ddns Yes if you have servers and want to use real domains

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Need to Extend My GnuPG Keys Again

Sun, 11 Aug 2024 00:00:00 -0400

I did not know what to post next.

What to do in this case?

Delegate and I ask a friend to pick up the next post ;-).

Thanks @shom for this pick!

The Context

My GNUPG Usage

I use GnuPG for many things:

encryption: I use passwordstore to encrypt some of my “system” passwords, for example my imap or smtp passwords.
signature: I sign all my commits, got that habit a while ago.
authenticate: I use gpg ssh-agent to authenticate everywhere, from regular ssh connection, to git write access.
decentralized proof of identity: After keybase’s purchase, I switched to keyoxide decentralized identity proof.

hardwark keys / yubikey ATTACH

Before I get into how I created my keys, I need to describe an important part of my GnuPG setup.

Since I dont fully trust my systems, I use Yubikeys as HSMs instead of storing my keys on my systems.

For people not familiar with HSM or yubikey, they are small computers packed in a usb stick or even smaller.

I’ve got two Yubikeys:

Yubikey 5 nano: The nano is always plugged in my laptop. It’s so small, it’s only protruding a small 1-2mm (or 1/8").
Yubikey 5 NFC: The intent for the YK 5 NFC is to use with my phone. It’s still a work in progress.

The idea is even if my systems are compromized, they can never read the keys. And worst case, I can change them with the locked down certify key.

There are cheaper alternatives to the YK5 NFC, but I haven’t found any for the nano form factor.

Couple of particularities with Yubikeys and similar HSM:

they require a user pin to be unlocked
if the wrong pin is entered n times (default 3), the keys stored are no longer usable unless unlocked by the admin pin.
If you enter the wrong admin pin n times (default 3), the keys stored are no longer usable at all. You must reset the yubikey and start again, or restore a backup.
You can enable touch confirmation when using the key. For example, to use the key, you enter the pin, and must touch the device for it to allow the operation.
They support more than just GPG keys, but I dont use any other features myself (like FIDO).

How I created my keys

My key workflow is quite similar to the one described in this debian post:

one never expiring certify key, locked down, never used except for managing keys.
sub keys created from the certify key, with a limited time validity.
extend or create new sub keys when they expire.

Here is the high level process:

I boot into a live distro
I created a main certify key in a locked down box (live usb distro), without internet. that key is assumed to be valid until I dont trust it anymore.
I then created a encrypt, authenticate, and signing sub2 keys from that certify key The sub keys are only valid for 2 years.
I backup all my keys to encrypted flash drives, which I store both in my house, and my parent’s house.
I copy the public keys on a readable flash drive.
I only transfer the subkeys to the yubikeys. The certify key only resides on encrypted flash which I never mount in my regular systems. And I extend them when they exif I still trust them.

The Problem

With all that said, my sub keys expired on August 8th… I discovered the issue when I tried to push my dotfiles changes to git.sr.ht over ssh.

I have reminders, but somehow I did not notice :).

The solution?

The process is not super simple, as there are so many ways to do this.

When I started using my Yubikeys, I discovered this amazing guide: DrDuh YubiKey Guide

Yes, there is a lot of reading, but the process is quite well described there.

In the case of a renewal, it’s much simpler though:

Boot a live distro: I used the live image using DrDuh nixos live flake this time
Mount the encrypted backup of my keys (including the certify key)
Load the keys in GPG
Edit each sub keys and set a new expiration date
Create a new encrypted backup of the keys
Backup the updated public keys
Update the keys on the Yubikey (not necessary for extending)
Boot on my regular system
Import my updated public key
Send my updated public key on my Web Dir, and on the default key server.

You can discuss this post on mastodon: https://fosstodon.org/@benoitj/112947067988523554

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Emacs as My IDE - Part2: The Setup

Thu, 08 Aug 2024 00:00:00 -0400

You can read part1 here.

Elisp configuration can be found here.

In part2, I’ll go over the setup to be able to experiment, and capture my findings.

LSP or not?

The objective is to try to use Emacs for real professional development including for languages like Java, go, or Python. My field of work requires me to quickly switch to other languages, now or known.

Emacs is great for some languages, especially LISP, or c and c++. For other type of languages, Emacs basic language modes can’t offer the same level of support as Language specific IDEs from JetBrains or Eclipse.

Only with LSP does Emacs can offer better language support.

So LSP it is.

Eglot or lsp-mode?

There are two mature LSP client implementation for Emacs: Eglot and lsp-mode.

lsp-mode is the oldest of the two, has a lot of features, for a lot of languages. It is extended by lsp-treemacs, lsp-java, dap-mode, and consult-lsp. It’s UI is quite IDE like, and diverge a bit from Emacs native practices.

Eglot, is the newer kid on the block, simpler. It is extended by eglot-java, dape-mode and consult-eglot. I know it less, so maybe there are features that I get with lsp-mode that are missing.

Before dape-mode, the choice between Eglot and lsp-mode was much simpler (lsp-mode), now Eglot is a plausible choice.

I would like to use Eglot for a more natural UI, but at the same time, if an important feature is missing, I will use lsp-mode.

At this point, I cant chose one over the other, so I will evaluate both.

One realistic project

To try and demonstrate features from the various I need a realistic project.

Here are some of the criteria to pick a project:

complex enough code base with many modules
contains automated tests
enough dependencies to test navigation and definitions
back-end application, this is what I see the most
Language I often see with enough complexity (Java)
Using frameworks I often see

In my work, I often see back-end Java projects based on spring boot.

Spring project has the spring-petclinic, a spring boot example project. It contains both a web front-end and an API back-end. This should have plenty of code, tests, and dependencies I can use in a realistic way.

Configuration Setup

I will build two new configuration, one based on Eglot, one based on lsp-mode. You can find the code here.

Shared configuration, like completion framework, will go in shared.el which is loaded by both configuration.

The share configuration will be minimal, yet, complete enough to enable my regular coding workflows.

Shared configuration that will be included:

navigation and editing helpers
completion at point
minibuffer completion
Database and REST client

What it will not contains:

org-mode configuration
email configuration
irc and other social network
any other modules unrelated to my workflow in IntelliJ

I will build the shared configuration when needed.

Loading bookmark list on startup

Loading a dynamic UI at startup is not that trivial. It works as expected for file, static or generated buffers, but not so much for dynamic UI.

Here what works:

(setq inhibit-splash-screen t)

(add-hook 'after-init-hook
          #'(lambda ()
              (require 'bookmark)
              (bookmark-bmenu-list)))
(setq initial-buffer-choice (lambda () (get-buffer "*Bookmark List*")))

As you can see above, you must launch the command that generate buffer for initial-buffer-choice to work.

The bookmark list itself is hard-coded in the shared configuration by setting the value of the bookmark-alist variable.

How to distinguish both configuration?

I decided to simply use the provided modus dark theme in the eglot configuration, and modus light theme in the lsp-mode.

Also, the bookmark list is altered to only show the relevant configuration.

Other minimal UI tweaks

I’m only planning to prompting, and audible bell.

1
2
3

(setq use-dialog-box nil)   ; use message prompt instead of popup dialogs
(defalias 'yes-or-no-p 'y-or-n-p)
(setq visible-bell t)       ; flashes instead of beeping

What’s next

I’m quite excited to continue this series.

Here is some planned content to expect in future posts:

Editing and navigation
Documentation, linting
Refactoring and code actions
Runners: building, tests, application, and Debugging
Supporting tools
Framework support

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Learning to Learn

Tue, 06 Aug 2024 00:00:00 -0400

Here is a simplified version of how I learn:

The sponge phase I document myself about a specific subject (documentation, blogs, books, YouTube, people. This phase is intense, kind of obsessive. At that phase, it’s about absorbing as much and as various information without thinking too much. I try to not make an opinion, I just absorb.
The experiment phase At this phase, I try stuff. If it’s coding back-end, then I try to build couple of simple APIs, simple tests, simple integrations. If it’s home improvement, I may create mock-ups, try with scrap wood. I try to define small and specific learning objectives. I don’t create complete working POCs, but just small snippet of things. This phase starts before the end of the sponge phase, when I have enough information.
The critics phase I look back at what I have learned, look back at my sources, and form my own opinion of what is good, bad. This is when I start to be critic of my sources, I trust no one. Lots of people forget this phase, and trust what they read, see, hear.

I apply this in most of my learning in my life, tech or not.

How do you learn?

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Native Emacs Bindings

Sun, 04 Aug 2024 21:03:42 -0400

Context

My journey with Emacs starting around 1996, while working on a cross platform portfolio management application, especially working on cross platform Windows / Linux GUI based on WxWidgets.

Emacs was great for C++ projects, even at the time. My configuration was minimal and I was using native bindings.

Then around 2000, I started coding in Java, and also discovered vim. Modal editing really clicked for me, and Vim became my editor for couple of years. I was still using Emacs on and off when I was coding in C or C++.

My journey with Emacs took a turn in 2016, while changing job, I rediscovered Emacs, this time with Spacemacs, and then with DoomEmacs. Both of these configuration makes extensive use of Evil, a better vim layout for Emacs :).

I continued using Evil, even with my “from scratch” configuration.

Except for couple of months sometimes in 2022, I’ve been using vim bindings even in Emacs.

The plan

Now is time to try something new, try to use Emacs native bindings and solve problems / inefficiencies that can be with the Emacs bindings.

Some of the discovery:

my use of <space> prefix can easily be replaced by C-c prefix. This became my “global” prefix for a while, and <space> when in normal mode.
I can use the C-c prefix for many of the annoying bindings for useful commands.
Avy, Embark, Ace Window can be used in combination to help me move around and improve navigation.

So I’ve been working:

first to replace general and all my global bindings, including use-package :general configuration with C-c prefix and the use-package :bind clause.
turn off evil and any evil related packages.

So since a week, I’m back on native bindings.

How it’s going so far?

I’m not too lost, but I need to develop navigation reflexes with avy and isearch.

Oh, and I need to go back to isearch, consult-line is not a valid replacement for isearch without the vim / search key available in evil.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Fixing Paving Mistakes

Sun, 28 Jul 2024 00:00:00 -0400

Story and some info about how I fixed some pavers installation miscalculations.

This post has nothing to do with computers, but everything to do with learning, and building things, skills quite related to coding and computers.

The Context:

Some time ago, I built a large pavers patio to put a dining table and gazebo.

Here is the initial work:

Figure 1: Picture of my backyard, with a gazebo sitting on a pavers patio

At the far back of the picture, we can see a sidewalk. That sidewalk connects the pool deck, the patio, and stairs that goes in the house.

Figure 2: another angle of the backyard

The Problem:

So what is the problem?

Figure 3: picture showing the sidewalk, and the stairs with visil

Figure 4: another picture showing the sidewalk, and the stairs with visible miscalculations

Two miscalculations:

When I calculated the width of the sidewalk, I totally forgot to account for the wood step that would sit on the sidewalk. That made the L shaped sidewalk much narrower on one side by roughly 1 foot or two rows of pavers.
I had to change the wood platform dimensions, so the sidewalk was also too short by about 1.5 foot.

The solution:

So last Sunday, was searching for something to do, I remembered another incomplete house crafting project: fixing that pavers sidewalk.

Armed with my shovel, a rake, my rototiller, a manual tamper, gravel, extra pavers, and some sand, I just fixed that problem.

Like many things, the base is the most important part. Usually you dig for a base of 6 to 12 inches, put some geo-textile material to prevent weeds to establish, build the base with gravel in layers tampering it regularly, build a much finer top 1/2 inch layer to put the pavers on, install the pavers, then tamper them again to lay them solid, put sand in the voids between the pavers, wash, and voila.

In this particular project, the challenge is to line the older part with the new part. also, because the older part was tampered with a mechanical vibration plate, it is much more solid, so the new part will sink a bit.

Here is the final result after 3 hours of work, you can see in the picture now there are always 7 rows of pavers visible, and the stairs sit on pavers for the full length:

Figure 5: picture of the final result, showing pavers sidewalk and the stairs sitting on it properly.

Other thoughts

This was one of many small projects / improvements I had on my list.

Often we think the main ingredient to do home crafting projects is time, in my case it’s usually focusing on something that we stopped seeing.

Last year, my partner and I made a list of things around the house that needed refresh, fixing, or completion, and I’m kind of refusing to start new cool stuff before the list is done.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

RSS, the best way to publish posts, and give control to your readers

Tue, 09 Apr 2024 00:05:08 -0400

So you’ve got a new shiny blog, or an old one, here is what I find important about RSS.

Here is my take on RSS

it’s the simplest way to automate notifications to regular readers
You give control to your readers, they choose how they read, when they read and in the format they read.
Some users use RSS not just for blogs, but for watching Youtube, peertube, reading Reddit, Slashdot, and more. They build a one stop shop for reading and watching medias.
It does not help discoverability, there are other options for that:
- Other people’s recommendations (this is one of my favorite way)
- blog rolls, blog rings (see craftering for an example)
- RSS aggregator (planet emacs, planet clojure, etc…)
- (In)frequent message on fedi about your new shiny posts

Some important things to consider when creating a RSS feed

Use a RSS web validator to confirm your feed is valid.
Include your complete post in your feed instead of just the title. It’s way better for your reader not to have to go to your website to read your post. There are some workarounds for web based readers, but it’s not accessible, not portable for text readers and not always compatible.
Enable discoverability of your feed from your website. You can do so by adding a <link rel="alternate" type="application/rss+xml" title="Example Feed" href="http://example.com/feed/" /> to your <html><head>...</head> tag. This will allow RSS readers to find your feed automatically by just looking at the index page of your site. You can have multiple <link> if you have multiple feeds.
Subscribe to your own feed, this way you will find out if it’s working as expected. This has the added benefit to get you used to RSS if you’re not already using it.
Make sure you do not move your posts / change URLs all the time. Use HTTP redirect if you want to move things. That is even more important with RSS feeds.
Make sure your RSS feed url is the same, use HTTP redirect if you want to move things.

What is next

Consider creating a RSS feed for your blog
Or if you already have one, maybe it’s a good time to see if you can improve it
And you may want to subscribe to my blog using RSS :-)

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

RSS Exclusive - Reminder on the RSS Club

Mon, 08 Apr 2024 23:39:29 -0400

Pssst - this is a RSS only post. Please keep it secret.
Read more about RSS Club: https://daverupert.com/rss-club/

I won’t give names, but during a public chat, one of my readers mentioned my “RSS Club” posts.

I don’t think anyone really noticed, it was a really close one.

Reminder to read the club rules again, especially you who I won’t mention your name :-D.

Joke aside, pretty cool to see people still using RSS. I use it more than ever. Going to try to have more RSS only posts in the future.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Back on Nix

Tue, 02 Apr 2024 12:39:02 -0400

Back on Nix

I’m back on nix. Guix / guile felt nice, here is why i’m back:

I’m really missing a practical reproducible OS, kept having issues with dependencies of my personal channel. This is something trivial when using Nix flakes. With Guix channels being a local configuration, and having to hack it to get it to work the way I would want to, this process of “dependencies of my dependencies” is much harder to fix.
Packaging hell: I wanted to package one of the rust tool I use, I cant easily point to a specific version without packaging every dependencies of that rust package. Guix is missing a way to easily package different versions of libraries, and we are always at one commit away from breaking a package.
I was still using nix for everything Guix does not fix or cover.
Spending time on my config instead of coding real stuff.
Need a different setup for work

For all these reasons, I got tired of Guix, even if I want to like it, it’s not meant for me.

Note to my future self: Before going back to Guix (next year?), remember this series.

Nix makes things work for me today, Nix flake keeps it working forever.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Guix After a while

Thu, 04 Jan 2024 01:53:26 -0500

It’s been couple of months after I migrated back to Guix.

Overall, the basic experience is good, comfortable. At the same time, some of the pain points, are quite painful:

Guix does not handle well dependencies between software from different channels
Some useful packages are still missing
For clear reasons, packaging anything golang, rust, java, javascript, and similar is pretty much unrealistic

Guix does not handle well dependencies between software from different channels

Guix aim to be fully reproducible, but that is one big missing part. 3 Times I could not build my config due to incompatibilities between Guix, NonGuix, and my channel. Due to my time-machine “hack”, I was able to recover an broke config relatively quickly, but this is a pain. One more reminder that I miss flakes.

Possible solution would be to code something like flakes is Guix, but I dont have the time to contribute to that level.

Some useful packages are still missing

Hugo, mopidy w/ many plugins are missing. restic is outdated. NerdFonts are missing (license).

Many lsp servers are missing.

I use nix a lot on top of my basic Guix setup.

Packaging golan, rust, python, java, javascript and similar apps is unrealistic

Primarily for these reasons:

amount of dependencies they have
dependencies and version specification in native language build files ignored
most libraries are:
1. in one version only
2. or missing
3. or one step away to be updated and break the software

I tried “hacking” it by using the gnu build system, and calling the build tool to fetch dependencies, but the “home” under build is read only.

I understand why it’s done this way, but it creates unecessary fatigue for something as self contained as a golang or rust app.

It would be possible to do it, but would require symbols to always contain a version, and we are not there at all. I needs everyone to do it, which means it wont happen.

Another example, I used chezmoi in the past, it was working until it did not due to one of it’s dependencies got upgraded.

For applications it makes sense, for core stuff it make sense, for apps dependencies, makes a lot less sense if they have a strong dependency management system.

In some ways, both will tend to break over time, but leveraging the work of upstream dev reduces the workload on packagers.

Final notes

Some observations of my current migration:

Always fun to go back to Guix
Guix still feels much less mature
Some of the core ideas in Guix make me use nix quite a bit
I’m missing flakes
guile scheme is much nicer than nix lang
I realize that I prefer to code apps than to code OS
I also realized even if scheme is great, Guix is not the right fit for my needs at this point
got to plan my next migration, I split my config to enable easier switch between nix and guix

You might have guessed, when the time is right, i’m back on nix.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

RSS Exclusive - App Defaults (2023 Edition)

Thu, 07 Dec 2023 18:20:05 -0500

Pssst - this is a RSS only post. Please keep it secret.
Read more about RSS Club: https://daverupert.com/rss-club/

I’m a fan of RSS and on my feed recently, I’ve seen “App Defaults” seven times.

Here is my take on it.

Operating system:	Desktop: nix, guix; Server: ubuntu
Window manager:	dwm
Web browser:	firefox, qutebrowser
Web feed client:	elfeed, miniflux
Search engine:	duck duck go
Bookmark manager:	links in org-mode notes
Email client:	notmuch
Email provider:	mxroute w/ my own domains
Calendar:	google agenda
Contact book:	google contacts
CalDAV/CardDAV provider:	don’t use
Instant messaging:	Desktop: ERC, Emacs/Ement; Mobile: Revolution IRC, Conversation; IRC Bouncer: soju
Audio and video calls:	Matrix, jitsi.meet, SIP phone
Cloud storage:	self hosted Truenas
Password manager:	bitwarden, passwordstore
Terminal emulator:	st
Shell:	bash
Text editor:	Emacs
Word processing:	Emacs, Libreoffice
Code editing/IDE:	Emacs, IntelliJ
Notes:	Emacs denote
To-do application:	Emacs org-mode
Music player:	mopidy
Video player:	mpv
Audioblog client:	same as my web feed clients
File manager:	Emacs dired
Photo management:	seeking for one. files and folders
Camera gear:	My phone
Budgeting/finances:	ynab, transitioning to beancount
Shopping list:	Emacs org-mode
Meal planning:	Low tech paper

App Defaults from others

Here are some people with RSS feeds I subscribed to which posted App Defaults recently:

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Progress on my Guix yearly migration

Wed, 01 Nov 2023 18:30:29 -0400

This is my first post since I started my Nix -> Guix migration.

TLDR: I had a basic system and home setup in 3 hours using my older configuration, but due to a kernel or grub bug, I lost 3 days. I found a workaround and now have a working system and home configuration for most things.

Some Issues and discovery

The first issue I had was a confusing one. And made me lost 3 days. In version 5.6 of both libre Linux and the Linux kernel, during boot, grub overlay no longer shows the password prompt to unlock the /boot partition. This issue is not found when using both kernels at version 5.0.

It took me 3 days to realize it was waiting for my input, but not showing the prompt. I believe I was running 5.6 on nix with the same kind of setup without any issues. Looks like a Guix specific issue at this point, or a problem found in Guix’s version of some software (Grub?). Something to follow-up on later.

Another issue happened while restoring my restic backup, I found out Guix’s version was really behind and incompatible with the backup created on nix. I ended up enabling nix-service-type and get restic with nix-shell -p restic. Also something to follow-up on.

Couple of interesting findings:

guix search command is much better and simpler to use than nix search
Guix ‘R us toy search engine is really good, much better than Guix’s web package search. It’s much faster, and allowed me to find a many 3rd party “home” services I was missing in Guix
I found out I created custom service-types and packages last year, I forgot. Less work.
I could use RDE’s service types for my email setup and for other needs when not found in Guix
As soon as I had nix-service-type working, my software anxiety went away.

My current state

I have a working system for many of the core things. I have a working desktop environment with slim, my custom dwm and related packages, and most of my core tools.

Hardware wise, I’ve got a “real” Linux kernel to get my WiFi working and recent ucode CPU patches. Also my bluetooth setup is functional, and the cups print server working.

My software safety net is setup with nix-service-type.

I have a flake like pinned channel setup pulled from RDE’s setup.

My system configuration and home configuration are setup right, and many software are configured by service types:

Emacs
email (goimapnotify, msmtp, isync, notmuch)
bash configuration
direnv
nix channels and nix-shell configuration
gpg, and ssh agents setup
git configuration

With nix’s restic, I was able to restore my home folder.

And finally, with nix again, I loaded Hugo and was able to write this post as Guix does not have Hugo.

You can look at my dotfiles and channel repos.

Next steps

My current main focus is to stabilize my system. I did a lot, but this left a big pile of messy code.

Couple of things to fix, cleanup and complete:

Cleanup and fix my notmuch tagging. Something is wrong and emails do not reach my tag:inbox.
Review the list of tools and adjust based on my previous research (see previous posts)
cleanup my config
update my channel with packages and services I have in my dotfiles repo
update my channel software versions (it’s 1 year old)

This should not take me more than 1 week. Need to time box this. I don’t need perfect, I need better.

Once I’m done with stabilization (and only then), I’ll improve my setup.

My first improvement will be related to my coding setup. Guile scheme, Clojure, Java, Node/JS, diagram tools.

Then a second improvement is to update my setup for contributing to Guix, especially test the email patch setup, a local clone of guix, the REPL.

Then it’ll be time to take another look at what 3rd party channel offers and improve my configuration.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Guix Home Configuration - Part 2 - Services

Tue, 24 Oct 2023 00:00:00 -0400

Nix home-manager

Nix home-manager has a lot of services. In fact, I don’t think I manage any configuration files myself except for my Emacs configuration. From daemon, to email, to bash, it’s covered.

In nix term, a program is a package + configuration, while a service is a program with an autorun.

email configuration

Configure account specific information. From email addresses to imap or smtp server details. This information is reused by many mail related tools.

nix	guix	comment
accounts.email.accounts.*	manual	configure email accounts w/ user/pass, email addresses, etc
accounts.email.accounts.*.imap	manual	imap conf for account
accounts.email.accounts.*.smtp	manual	smtp conf for account
accounts.email.accounts.*.msmtp.enable	manual	send email with msmtp
accounts.email.accounts.*.imapnotify	manual	enable goimapnotify support
accounts.email.accounts.*.notmuch.enable	manual	enable notmuch for account
accounts.email.accounts.*.mbsync	manual	enable mbsync for account

X11 specific config

These elements configure X and how to load X with Xsession.

nix	guix	comment
fonts.fontconfig.enable	unclear if I need this-command	enable font config
xresources.properties	home-files-service-type (manual)	configure xresources and load it with k/v
xsession.enable	home-files-service-type (manual)
xsession.initExtra	home-files-service-type (manual)
xsession.windowManager.command	home-files-service-type (manual)	command to run to start my window manager

programs

Programs in Nix are used to install and configure applications or tools.

nix	guix	comment
programs.autorandr.enable	autorandr
programs.autorandr.profiles	missing: install config	map of profiles and config
programs.bash.enable	bash-service-type	enables bash. `.bash_profile` loads `.bashrc`, default `.profile` loading home managers session vars
programs.bash.enableCompletion	home-bash-configuration bashrc (manual)	enables completion
programs.bash.initExtra	home-bash-configuration bashrc	adds to `.bashrc` for interactive shells (after check interactive)
programs.bash.profileExtra	home-shell-profile-configuration	adds extra at the end of `.profile`
programs.bash.sessionVariables	home-environment-variables-service-type	adds extra exports to `.profile` using k/v map
programs.bash.shellOptions	home-bash-configuration bashrc (manual)	adds shopts to interactive `.bashrc` using k/v map
programs.bash.shellAliases	home-bash-configuration aliases	adds aliases to `.bashrc` using k/v map
programs.bash.logoutExtra	home-bash-configuration bash-logout	adds extra to `.bash_logout`
home.sessionPath	home-environment-variables-service-type (manual)	add dir to `PATH` in `.profile`
programs.direnv.enable	direnv w/ home-bash-configuration bashrc (manual)	enables direnv in `.bashrc` interactive
programs.direnv.nix-direnv.enable	home-file-service-type for `use guix`	from guix cookbook
programs.emacs.enable	I will install my config with home-files-service-type	enable configuration of emacs
programs.emacs.package	pick the right emacs 29 + nativecomp	which emacs package to use
programs.emacs.extraPackages	extend the package if needed	which extra nix packages is added to the dependencies / visible to emacs. the rest I pull with straight
programs.git.enable	home-xdg-configuration-files-service-type (manual)	configure git
programs.git.package	home-xdg-configuration-files-service-type (manual)	package to use
programs.git.userEmail	home-xdg-configuration-files-service-type (manual)	email
programs.git.userName	home-xdg-configuration-files-service-type (manual)	user name
programs.git.aliases	home-xdg-configuration-files-service-type (manual)	aliases k/v map
programs.git.attributes	home-xdg-configuration-files-service-type (manual)	attributes k/v map
programs.git.difftastic.enable	home-xdg-configuration-files-service-type (manual)	enable difftastic
programs.git.signing	home-xdg-configuration-files-service-type (manual)	set signing key
programs.git.extraConfig	home-xdg-configuration-files-service-type (manual)	extra config k/v
programs.mbsync.enable=true;	home-files-service-type (manual)	configures mbsync
programs.msmtp.enable	home-files-service-type (manual)
programs.notmuch.enable	home-files-service-type (manual)
programs.notmuch.new.tags	home-files-service-type (manual)
programs.notmuch.search.excludeTags	home-files-service-type (manual)
programs.notmuch.maildir.synchronizeFlags	home-files-service-type (manual)
programs.notmuch.hooks	home-files-service-type (manual)
programs.ncspot.enable	missing, nix for now	configure ncspot
programs.ncspot.settings	home-xdg-configuration-files-service-type (manual)	k/v of settings.
programs.password-store.enable	password-store	enables
programs.password-store.settings	home-xdg-configuration-files-service-type (manual)	k/v
programs.qutebrowser.enable	nixpkgs	since qute on guix does not work for all that I use
programs.qutebrowser.loadAutoconfig	home-xdg-configuration-files-service-type (manual)
programs.qutebrowser.extraConfig	home-xdg-configuration-files-service-type (manual)
programs.qutebrowser.keyBindings	home-xdg-configuration-files-service-type (manual)
programs.rofi.enable	rofi	configure rofi
programs.rofi.font	home-xdg-configuration-files-service-type (manual)	font to use
programs.rofi.theme	home-xdg-configuration-files-service-type (manual)	theme to use
programs.rofi.terminal	home-xdg-configuration-files-service-type (manual)	terminal to use
programs.rofi.extraConfig	home-xdg-configuration-files-service-type (manual)	extra config to add
programs.starship.enable	found in 3rd party channel	adds starship setup to `.bashrc`
programs.starship.settings	home-xdg-configuration-files-service-type (manual)	i disable stuff

services

Services are programs which are autolaunched in various ways: shepherd, from Xsession, from a startup script.

nix	guix	comment
services.syncthing.enable	home-syncthing-service-type	start syncthing user daemon
services.gpg-agent.enable	home-gpg-service-type	enable agent
services.gpg-agent.enableScDaemon	there is a system service, unclear what else	enable support for yubikey
services.gpg-agent.enableSshSupport	home-gpg-agent-configuration ssh-support?	enable gpg agent as ssh agent
services.gpg-agent.sshKeys	home-openssh-configuration	specify keys to load
~~services.spotifyd.enable~~	abandon
~~services.spotifyd.settings~~	abandon	k/v
services.imapnotify.enable	home-xdg-configuration-files-service-type (manual) w/ shepherd
services.mbsync.enable=false;	home-files-service-type (manual)	since I start mbsync with imapnotify
services.dunst.enable	home-files-service-type w/ xsession.initExtra (manual)
services.picom.enable	home-files-service-type w/ xsession.initExtra (manual)
services.sxhkd.enable	xsession.initExtra (manual)
services.sxhkd.keybindings	home-files-service-type w/ xsession.initExtra (manual)	k/v
services.dwm-status.enable	xsession.initExtra (manual)
services.dwm-status.order	home-files-service-type w/ xsession.initExtra (manual)	array of info to see
services.dwm-status.extraConfig	home-files-service-type w/ xsession.initExtra (manual)	k/v
services.screen-locker.enable	xsession.initExtra (manual)	enables auto locking
services.screen-locker.lockCmd	home-files-service-type w/ xsession.initExtra (manual)	process to start when locking
services.poweralertd.enable	home-batsignal-service-type
services.unclutter.enable	home-unclutter-service-type

Other configuration

Where I want to supply my own complete dotfile, I use home.file to install it. In nix, it’s really just for few.

My strategy for getting Guix running ASAP for missing services or programs is to capture my current configuration files and install them with a similar guix home service.

nix	guix	comment
home.file.abcde	home-files-service-type	copy dotfile
home.file.emacs-init	home-xdg-configuration-files-service-type	basic home file copy of my emacs init file to `~/.config/emacs`
home.file.local-bin	home-files-service-type	copy my small shell scripts to `~./local/bin`
home.file.lynx	home-xdg-configuration-files-service-type	copy dotfile
home.file.surfraw	home-xdg-configuration-files-service-type	copy dotfile

3rd party services

I found couple of services like dunst in 3rd party channels. Once I have a working setup, I should be looking at them + RDE which has a lot of good home services, on top of writing my own.

What’s next?

I know this works and can work.

I’ve got solutions for everything, the next steps are:

backup config files generated by nix that I need to now manually handle
backup files for reference: bash config files + xsession
backup my laptop
install!

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Guix home configuration - part 1 - Packages

Mon, 23 Oct 2023 18:00:27 -0400

The most important part of my Nix configuration: nix home manager.

From packages installed, services enabled, configuration copied or generated, home manager is where the Linux “customization” happens.

Nix flakes alternative

Lets make this simple, my primary flake is my dotfile configuration.

No matter the setup, both Nix and Guix rely on the source control to make sure the packages are reproducible. When you pull on sync your channels, Guix (or Nix) “pull” changes to a local copy of the repository.

In functional expression, a flake derivation would look like this: f(pinnedInputs, configuration) = configDerivation where pinnedInputs are the channel + commit hash for each flake (channel) specified and stored in git.

This makes the configuration flake, a function of it’s input. It’s reproducible, there are no external dependencies.

In Guix, the pinnedInputs exists, but are external and kind of invisible to the user. They represent your last “pull” command, nothing more. It’s place sensitive and not captured anywhere but on your local machine.

What it means in practice?

Without this critical information, re-installing the configuration will produce different results, unless Guix (or Nix) did not change since last time you did it.

For example, last time I migrated to Guix, using my previous migration’s configuration, I could not get chezmoi dotfile manager to work. It failed to build between the time I left Guix and the time I re-installed Guix. I had spend time figuring out when the build started to fail, and wasted time getting my system up to date.

The solution for this is to store both the channels used by the configuration as well as the git hash versions of each channels. When they are in your repository, you have a lot more chance to get a working system, especially after some time. It also make it super easy to upgrade one channel while keeping an older version of another one by just undoing your changes using git.

When you are a coder, this is super natural to do. Many of the development tools actually have similar mechanism (like npm’s package.lock).

how do we achieve that with Guix?

Guix does not have an off the shelf pinning system, but has some commands to support similar workflow: describe and time-machine.

On any Guix system, if you run guix describe --format channels, you get the same info as your channel file, but with the specific git hash from the latest guix pull. We could store this output into a file in our configuration git repo for later reference.

The 2nd useful command is guix time-machine. It allows you to run a Guix command for a specific version. There are many ways to specify the version, one of which is a channels file. We can use the output of guix describe to run guix pull at a specific version using guix time-machine.

The is the closest we can get from Nix flakes.

The main limitation is that Nix flakes tracks inputs of inputs, while guix describe/time-machine will only track your immediate channels specifications.

This is the same mechanism that RDE uses (look at their setup here).

Software packages

Here is an inventory of all the software I use, except for anything related to nix home-manager (programs and services).

I’m been using whereiseveryone’s toy search engine. It index Guix main channel, but also many other 3rd party channels. The search engine is not just more complete, it’s also much faster than Guix packages web search engine.

In my case, most will come from Guix’s channel, some from NonGuix, and the rest from my own channel.

Note: before installing any software from 3rd party channels, it’s always advised to inspect the Guix package code, and even better, inspect and replicate some of them into your own channel. You should not leave security to luck. Since I only have couple of packages in unknown 3rd party channels, I’ll inspire myself and update my own channel.

Tools

Shell and core tools

nix	guix ([channel#]<package>[:output])	comment
babashka	bin-guix#babashka	issue is graalvm not in Guix due to reproducibility
bat	=
bash-completion	=
coreutils	=
dosfstools	=	dos partitioning/formatting tools
lesspipe	=
~~nix-bash-completions~~	missing, abandon	I dont really need it
starship	th#starship-bin	This could go in Guix, but really a lot of dependencies

network

nix	guix ([channel#]<package>[:output])	comment
bluez	=
bluez-alsa	=
bind	bind:utils	I use the lookup tools
curl	=
openssl	=
rsync	=
sshfs	=
surfraw	=
~~tig~~	abandon	I dont use this anymore
wget	=

File management and processing

nix	guix ([channel#]<package>[:output])	comment
(aspellWithDicts (dicts: with dicts; [ en en-computers en-science fr]))	aspell, aspell-dict-en, aspell-dict-fr	Note en-computers and en-science in kbg
	kbg#aspell-dict-en-computers, kbg#aspell-dict-en-science	Note2: aspellWithDicts equivalent?
~~atool~~	abandon	file archive manager I dont use
beancount	=
bsdgames	bsd-games
cowsay	=
~~dateutils~~	abandon	I dont use this
dos2unix	=
diffutils	=	diff, cmp, diff3
fd	=	find replacement
figlet	=	banner tools-extra
findutils	=	find, locate commands
fzf	=
gawk	=
gnuplot	=
gzip	=
highlight	=
jet	missing, nixpkgs	like jq, but include transit and edn
jq	=
ledger	=
mediainfo	=
p7zip	=
pandoc	=
pbzip2	=	# multi core replacement for bzip2/bunzip2
recutils	=
ripgrep	=
tree	=
texlive.combined.scheme-full	texlive
~~unrar~~	-> p7zip
unzip	=
wordnet	=
zip	=

Process management and monitoring

nix	guix	comment
acpi	=
htop	=
lsof	=
progress	=
pv	=	pipe progress monitor
strace	=
sysstat	=

Media tools

nix	guix	comment
alsa-plugins	=
~~alsa-tools~~	missing, abandon	I’m not sure I actually use this
alsa-utils	=
~~apulse~~	missing, abandon	not found but don’t use
ffmpeg	=
gst_all_1.gst-plugins-base	gst-plugins-base
gst_all_1.gst-plugins-good	gst-plugins-good
gst_all_1.gst-plugins-bad	gst-plugins-bad
gst_all_1.gst-plugins-ugly	gst-plugins-ugly
gst_all_1.gstreamer	gstreamer
imagemagick	=
mpv	=
pulsemixer	=
pamixer	=
sox	=
vorbis-tools	=

Security

nix	guix	comment
keychain	=	great tool to share ssh/gpg agents within a session
gnupg	=
pass	password-store
pinentry	=

Containers, images and virtualization, and deployment

nix	guix	comment
~~deploy-rs~~	missing, abandon	I used this for some of my servers. I have a diff solution now
docker-compose	docker, docker-cli, docker-compose	Unclear if compose is needed as it’s supported in cli on new versions
vagrant	missing, nixpkgs	I would want to use that to provision my work VMs

Programming tools

nix	guix	comment
ansible	=
autoconf	=
automake	=
binutils	=	ld and similar tools
black	python-black	python code formatter
cargo	=
clj-kondo	nonguix#clj-kondo	clojure linter
clojure	clojure, clojure-tools	this is new. was not in Guix if I recall
clojure-lsp	missing, nixpkgs	could not find. self package?
cmake	=
editorconfig-core-c	=
gcc	=
gh	ffab#github-cli
ghc	=
gitFull	git:out,send-email,gui
git-annex	=
~~git-doc~~	N/A
git-extras	missing, wait	lots of cool features. Maybe there is an emacs similar package. some are covered by magit
~~git-hub~~	-> github-cli
~~git-stree~~	abandon	I use magit
gitstats	magit-stats
gnumake	make
go	=
gopls	=	go language server
graphviz	=
html-tidy	missing, nixpkgs
jdk17	openjdk	21 is also available
jetbrains.idea-ultimate	small-guix#idea-ultimate
leiningen	nonguix#leiningen
libtool	=
maven	=
google-java-format	missing, nixpkgs	cli java formatter
mdl	missing, nixpkgs	markdown linter
mr	myrepos
nodejs	node	latest lts available (18)
nodePackages.stylelint	-> node-eslint
nodePackages.js-beautify	missing, -> npm prettier	replace with prettier from npm?
nixfmt	from nix package mgr
plantuml	=
pipenv	missing, wait
pyenv	missing, wait
python3	python
python-lsp-server	=
python beautifulsoup4	python-beautifulsoup4
python flake8	python-flake8
python isort	python-isort
python lxml	python-lxml
python mypy	python-mypy
python nose	python-nose
python pillow	python-pillow
python pyflakes	python-pyflakes
python pylint	python-pylint
python pynacl	python-pynacl
python pytest	python-pytest
python pyright	missing, abandon	some extra linting, not critical
python pypdf2	python-pypdf2
python terraform	=
python tldextract	python-tldextract
proselint	python-proselint
ruby	=
rustc	rust
sbcl	=
shellcheck	=
shfmt	missing, nixpkgs
sqlite	=
vscode-fhs	missing, wait	it’s a wrapper that allows to install vscode plugin normally
		is vscodium doing the same?

Tools perso

Tools I use only on my personal laptop

nix	guix ([channel#]<package>[:output])	comment
borgbackup	borg
borgmatic	=
~~bitwarden~~	-> rbw
~~bitwarden-cli~~	-> rbw
hugo	rosental#hugo-bin
rbw	=	bitwarden client
rofi-rbw	missing, nixpkgs	rofi frontend. easy to package
syncthing	=
transmission	=
yt-dlp	=
yubikey-manager	python-yubikey-manager
beancount	=
beancount-language-server	missing, nixpkgs
fava	missing, nixpkgs
hledger	=
~~chezmoi~~	abandon
profanity	=
kicad	=
qmk	missing, git clone	I may be able to clone qmk instead
abcde	=
beets	=
cdrdao	=
cdparanoia	=
cddiscid	missing, nixpkgs
chromaprint	=
dr14_tmeter	missing, nixpkgs
flac	=
fluidsynth	=
mopidy	missing, wait
mopidy-local	missing, wait
#mopidy-spotify	missing, wait
mopidy-somafm	missing, wait
mopidy-mpd	missing, wait
mopidy-iris	missing, wait
#mopidy-internetarchive	missing, wait
mpc_cli	mpd-mpc
mpd	=
ncmpcpp	=
ncspot	missing, nixpkgs
opusTools	opus-tools
python39Packages.pyacoustid	missing, wait
python39Packages.requests	python-requests
rubberband	=
sox	=
spotifyd	missing, wait
spotify-tui	missing, wait
twolame	=
wavegain	missing, wait

When missing, i’ll just use the nix package instead until I do something else about it.

Tools work

nix	guix ([channel#]<package>[:output])	comment
awscli2	awscli
azure-cli	missing, nixpkgs

Desktop

nix	guix ([channel#]<package>[:output])	comment
alacritty	=
arandr	=
autorandr	=
brightnessctl	=
cantarell-fonts	font-abattis-cantarell
dunst	=
fontconfig	=
dejavu_fonts	font-dejavu
dwm	mychannel#bmj-dwm
feh	=
firefox	nonguix#firefox
gimp	=
libnotify	=
libreoffice	=
light	=
~~kdiff3~~	missing, abandon	use emerge, or other diff tools
maim	=
mate.mate-polkit	mate-polkit
mpv	mpv
~~mupdf~~	-> zathura
nerdfonts CascadiaCode	mychannel
nerdfonts FiraCode	mychannel
nerdfonts Hack	mychannel
picom	=
~~projectm~~	missing, abandon
qutebrowser	nixpkgs	guix has a build that don’t work for me
rofi	=
~~scrot~~	-> maim
st	mychannel#bmj-st
sxiv	=
tesseract4	tesseract, tesseract-*	I dont need this now
widevine-cdm	missing, nixpkgs	needed for qute
wmctrl	=
xclip	=
xdg-utils	=
xdg-user-dirs	=
xdotool	=
xdragon	dragon-drop
xlockmore	=
xorg.setxkbmap	xkbset?
xorg.xev	xev
xorg.xinput	xinput
xorg.xrandr	xrandr
xorg.xrdb	xrdb
xorg.xset	xset
xsel	xsel
zathura	zathura, zathura-*

what’s next?

My program.* and services.* nix configuration -> service-types mapping.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Guix VM System Configuration

Sat, 21 Oct 2023 11:49:47 -0400

Something I forgot on my last Guix system configuration post, my work VM setup.

Obviously it’s built on top of my regular setup, but it has some specifics.

VirtualBox

The main difference between running on bare metal and running in VirtualBox is that I do need the virtualbox-guest-additions. This allows to:

have a seamless mouse / keyboard integration with the host OS
it also resize the desktop “screen” to the size of the window
It allows me to mount local host folders as drives on the VM
Has shared clipboard
allows to share USB devices (USB sticks and more) with the guest OS

It does require to install and load a kernel module. You might know where this is going. Guix by default does not support that.

The solution is simple though, since I use the Linux kernel, and nonguix build system, I just need to modify my kernel and create a package (or find one) for guest additions.

I found here a guix config that does what I need to the kernel and package guest additions.

Building the image

On nix, I used packer to build the nix image so I don’t have to go through the installation process.

I’m planning to build the image using guix system image command with my vm system configuration. This actually is a better situation than Nix, but I never tried it.

What else?

The only thing else special is auto-resizing the ext4 partition. It’s not infrequent that I have to enlarge the VM disk, having auto Resize set, enable me to just reboot the machine to benefit from the full space.

I think that’s it on the “system” front.

What’s next

You might notice I’m trying to post everyday. Like I mentioned on my first post, I won’t install Guix until I have a plan for everything. Tomorrow hopefully I have done the research needed for Guix home setup, and be ready to install Guix, who knows. Ah, I also need to do a backup…

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

How Guix Compare to Nix and Vice Versa

Fri, 20 Oct 2023 21:36:52 -0400

This started as a question from Mr. X on System Crafters general room on matrix: maybe I was offtopic here, but nobody who can compare Guix and NixOS?

This is my opinion, based on me running nix and Guix (and RDE) as my primary OS for the past 3-4 years now.

People who know me on the System Crafters community know I’ve switched between both almost as often as @acdw has re-started his Emacs configuration 🤣.

Take this with a grain of salt with regards to current state of Guix, I’m about to jump in the Guix wagon again but I don’t know what happened in the last 6-12 months.

Note that I use interchangeably the Guix and Nix package manager and OS, in this post, it does not matter.

Below, is re-worded version of the messages I posted at that time.

What do they have in common

Guix and Nix share a lot in common

They are functional package managers and OS. you can reproduce their build (some caveat), and roll back when you need to.
The concepts are the same, with derivations, a build daemon, packages that user can install, and where multiple versions can coexist. From what I know, Guix build daemon started as a fork of nix build daemon
Installation are typically descriptive with a file, instead of ad-hoc commands
They are both addictive
They both have opinionated, technically savvy communities

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Guix system configuration

Tue, 17 Oct 2023 08:51:23 -0400

In this post, I’m starting the analysis phase of migrating to Guix.

The first step is analyzing my current nix system configuration, and what I would need to do to port it to Guix.

My use cases

I run Nix, and soon Guix, on two types of desktops:

At least one work laptop running Linux in a VM where I:
- code, primarily in java, Node, JavaScript, Typescript, HTML/CSS, Python, Golang, and any other language that I encounter
- Review code from others
- run server applications for development and testing purpose
- access cloud accounts (AWS, Azure, GCP)
- Write technical documentation, in asciidoc, or markdown with various diagrams
- Browse documentation
- Capture notes and TODOs
My personal laptop, which has pretty much all of the above, with additional “fun” features:
- matrix, IRC, mastodon
- video and music streaming
- edit media files

software from my system configuration

The following is software part of my system configuration. They are really minimal, and required to get the system up. Some of these are needed on boot or requires special privileges.

nix	Guix	comments
((emacsPackagesFor emacsNativeComp).emacsWithPackages (epkgs: [epkgs.nix-mode]))	default emacs + scheme mode	this is not my full setup, but a minimal system configuration to be able to edit configuration files. I dont need native comp at the system config
libvterm-neovim	emacs-vterm	in nix, this neovim is the way to get the “so” for libvterm. useful for emacs vterm
nfs-utils	=	mounting nfs
git	=	this allows me to access my dotfiles repo
gnumake	make	my dotfiles need make to launch commands
gnupg	=	for smartcard setup
pinentry	=	for gpg setup. do I need additional pinentry-* packages?
lynx	=	decent text web browser
surfraw	=	quickly search the web
st	abandon	i’ll move this to my home config
~~vim~~	abandon	havent opened vim in ages. emacs is my vim
wget	=	simple way to download files
openvpn	=	system app
networkmanager-openvpn	network-manager-openvpn	integrates openvpn in network-manager
lxqt.lxqt-policykit	lxqt-policykit	polkit interface. I believe I need it for yk + gpg, but unsure
glxinfo	mesa-utils	describe video 3d capabilities
qmk-udev-rules	N/A, create config	I need to set thes udev rules myself. wondering if this could be a Guix package

system services enabled

I tend to take a similar approach to software for my services. I usually only have services that are not good fit for my home configuration.

nix	guix	comment
hardware.sane w/ airscan backends	sane-service-type	I use this to setup my scanner with airscan configuration
services.avahi w/ nssmdns	avahi-service-type	seems doing the same as nix
services.pcscd	pcscd-service-type	smartcard support (yubikey)
networking.networkmanager	network-manager-service-type	this is my go to network configurator
services.gvfs	package gvfs	usermounts for usb keys. could not find a service. unclear if i’m missing something
hardware.bluetooth	bluetooth-service-type	bluetooth service and tools
services.printing	cups-service-type	cups daemon redirecting to my network printer
services.fwupd	iptables-service-type	not exactly a match, but does the same result
virtualisation.docker	docker-service-type	container daemon
sound and hardware.pulseaudio	pulseaudio-service-type	sound daemon
services.xserver display manager	sddm-service-type or slim-service-type	I may need to modify my dwm package to install an Xsession file

Other system configuration

Additional configuration found in my nix system configuration.

nix	guix	comment
hardware.opengl	unknown	this enable opengl and dri with mesa. not critical early on, but speed up the UI
backlight control	udev configuration	udev rules to enable backlight control for the `video` group. maybe I can package this.
~~console.useXkbConfig~~		use my X configuration for console too. I dont really use this.
xkbOptions = “ctrl:nocaps”	set-org-configuration	should be able to set this on X11 system config
~~synaptics.enable=false~~		disable my touch pad. I do have another way to do this in my home config
hardware.cpu.intel.updateMicrocode	microcode-initrd from nonguix	cpu specific microcode updates. even if this is a blob, it contains important fixes and security patches
kvm-intel	unknown	enable kvm kernel for intel

Some of these config are specific to my intel laptop. may need to be split once I setup for both work VM and personal desktop.

Compare with my old Guix config for any missing bits

Here is my older Guix system configuration files.

The only thing I’ve got in my Guix configurations and not in my Nix setup: openssh server.

Coming next

I need to do the exact same analysis for my home configuration. I told myself I would not install Guix before i’m done with the preparation.

Stay tuned!

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Going Back Series - Guix Is Back

Mon, 16 Oct 2023 20:37:07 -0400

It’s Guix season again this year, I’m going back to Guix Linux.

My current setup with NixOS

Let me start by describing my current NixOS setup:

Configuration for my personal and work use cases
System configuration for my laptop and a work VM
Heavy use of home manager to configure and enable tools I need (desktop, coding, terminal, tools, and more)
Desktop setup with dwm, st, and many other tools
Enabled docker and qemu setup
YubiKey enabled
frequent use of direnv and nix-shell
Minimal custom code, beyond my config

Why not go back to Guix?

Ok this time, i’m starting with cons.

Obviously there are two sides to the Guix coin:

I’m can’t get all my work tools from Guix main channel (commercial)
sometimes versions lag or are missing due to packaging complexity. For example NodeJS was older for some time. Another example is Kondo and Babashka, both tools depend on GraalVM which is almost impossible to build from source at this point. Guix requires to build from source.
Smaller user base than Nix, means less packages, less home services
Default kernel does not support my hardware.

Why go back to Guix then?

Some of the key reasons:

I love LISP. What I learn is reusable. Not the same for Nix, the Nix language is only useful for Nix packages and configuration.
Many of it’s shortcoming are addressed
- Nonguix has many of my missing software
- Many other missing software are found in the whereiseveryone toy’s search engine
- I’m able to create new packages when needed. Got my own channel.
- If I have a urgent need, I still can install them using Guix’s nix-service-type or docker.
The System Crafters great community. I’ve been part of this community
I’ve got to install Guix for a while each year

The plan

Ok, what is the plan?

Objectives

First I need to make sure I have the least amount of downtime. I need to get my core desktop (dwm) and Emacs setup within 1 day.

Get my other missing software from Nonguix or other channels. Anything else provided by nix if possible.

I also want to document the process with short blog post ofter each day I work on this.

Steps

To minimize downtime and work, I will reuse my old Vanily Guix setup. My most recent Guix experience was with RDE, but I’m going to skip it for now.

One of the key first step will be to make an inventory of all the software I use in Nix, and compare with my Guix setup. I’m not too worried by software.

What will be more challenging is covering all my nix services. Guix does not come with many home services compared to Nix, I will either have to find missing services somewhere else, write my own, or just use a plain config to start with.

No time to waste, the Re-Guix journey begins (again).

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Going Back to Basis

Mon, 16 Oct 2023 19:26:46 -0400

Trying clojure blog generator

On my last post, I mentioned I changed my blog setup to cryogen and clojure.

While it was a mostly successful experiment, I’m going to keep it at that, an experiment. Going back to my hugo / ox-hugo setup.

My cryogen setup is still available here.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

New blogging setup

Thu, 29 Dec 2022 00:02:13 -0400

If you read this, it means I’ve moved away from hugo, to a fully custom clojure blogging setup.

The whole setup achieves pretty much the same features I had with hugo, with barely more than 100 lines of code.

You can read the code here.

The setup is roughly 100 lines of code that use asciidoctorj, selmer templates, and clojure maps.

I tried to maintain the similar features as before:

simple markup (asciidoc)
CSS support
able to generate diagrams out of plantuml or other diagram markup language
keep my older page location working while building a new structure

If you see any issues, let me know, this is freshly brewed :)

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Seeking a Hugo Blog Replacement

Thu, 15 Dec 2022 06:07:25 -0500

The path to replace Hugo.

My current blog setup always worried me a bit due to the complexity it has.

I write org-mode files, that are then exported to Hugo markdown file with the help of the great package ox-hugo, and these markdown are then converted by hugo to HTML and RSS.

That process works and is quite stable, yet, introduce complexity to:

understand how ox-hugo works
understand how Hugo generates it’s files
understand how to extend Hugo for my theme
understand the mapping between org-mode and the generated files

It is stable so I had little incentive to fix this complexity until now.

Clojure is my new hammer

Some “events” happened that made me consider replacing Hugo:

With all the fun I’ve been having using Clojure, I decided Clojure to be my hammer until I understand what it’s bad for.
I’ve completed 10ish days of AdventOfCode using Clojure and babashka (Clojure scripting)
Then I stumble upon (well I searched for it) some post mentioning cryogen, and then I found borkdude’s quickblog related posts (post1, post2).

All of this made me think that replacing Hugo with babashka is a brilliant idea.

After all, an org file is DATA :D

The plan

I write this hopefully using Hugo for the last time.

Here are some of the objectives:

keep things simple
either keep org-mode, or use asciidoc
1. org-mode is more and outliner than a markup language. Rendering/export requires emacs org export modules.
2. asciidoc is standard, has all the documentation features of org-mode, has good clojure/java integration with asciidoctorj, has a lesser editing mode than org-mode.
3. avoid markdown at all cost due to the need for extensions, but yet, no good standard.
reduce steps and necessity of having an intermediate step to generate my site. I want to get rid of the middle component in: org-mode -> markdown (or some markup) -> HTML/RSS
leverage babashka so I can build my blog using CI tool without needing to install Emacs. My blog is built by sourcehut’s build engine.
learn more about solving practical problems with Clojure
gain full control on the generation of the HTML and CSS
keep my clean semantic HTML5 generation and my current CSS

Non-goal:

write another minor mode like “ox-hugo”
modify my Emacs configuration
keep org-mode at all cost

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

RSS Exclusive - 2022w50 What Kept Me Busy This Year

Wed, 14 Dec 2022 21:59:24 -0500

Pssst - this is a RSS only post. Please keep it secret.
Read more about RSS Club: https://daverupert.com/rss-club/

Got busy this year

This will be a short post, but I’ve been busy this year.

The main reasons i’ve not posted in a while are due to personal events and all the efforts I spent with home improvement (also known as house crafting).

Will focus on “crafter” stuff.

Technology and Work

This year has been quite interesting in terms of technology, and coaching:

I started 2 projects
I kept an internal project alive
I now mentor 7 software architects (the kind that code)
couple of pre-sales
couple of practice activities, RACI with our PMO office, some security stuff
and a lot more

Next year should be a transition year for me, with more coaching. lets see.

Learning

Couple of the things I learned this year:

Kafka (I know many of the patterns, but used it for the first time)
Lot of integration work done with Spring Integration
A lot more of Clojure This is now drastically affecting the way I code in any language, FP or OO.
New business area (Natural and organic food distribution)

coming next

Few things on my plate next year:

A lot more clojure, I may actually create my next post using clojure.
A lot more coaching at work.
New projects i’m not aware of at this moment

House crafting

A lot has been going on this year in this area.

One of our main objective is to complete most of the house repair and improvement ASAP.

My own personal objective is to get it done sometime in 2023.

The Office

This year, and due to the pandemic and the “work from home” challenges, 8 months was spent building a new office from the ground up. A lot of challenges was also included

Here are some of the key criteria for my new Office:

use space in our house we do not use well, yet, not use any of the rooms we currently have (they are all occupied).
must be closed with a door
more important than a door, must be soundproofed, not so much for me, but for other occupants that are living in this house and suffer from me speaking all day on voice chat.
needs to be fully finished so that I can move my work and personal computers
can’t start a new house crafting project before this one is done
fix all the stuff that needs to be fixed where I build this office

We settled to a space in the middle of the house, which we dont use beside to go from one living room to a bedroom.

Here are some things that had to be fixed or completed prior to raising news walls:

fix a leak I thought was coming from a foundation crack. This happened to be my drain pipe cracked over 8 feet long.
fix a foundation crack
insulate a exterior wall
move water supply pipes to a reachable area
install a 30 amp power cable to the pool heatpump
replace the dangerous electric panel
move a lot of power outlets, switches, light fixtures to accomodate the new room I’m building

The office itself is just the right size, 8’ by 12’. It’s made of 2x6 walls with 2x4 studs staggered, insulation, sonopan, and multiple layers of 5/8 drywall, a solid wood door, and acoustic ceiling, and more.

Fixing stuff in my kids rooms

This one was simple, yet took me more than 1 month to complete. I had to repair one wall in each room due to previous unfinished repairs / project started years ago (contradicting our high level objectives :D). So mudding, replacing defecting convection heaters, painting, installing trims (another incomplete past improvement).

As of today, we’re only left to paint the trims (next weekend).

What is coming next

Few things on my plate next year:

Creating a bathroom in a place you cant call a bathroom anymore, and where everything is leaking, and is unusable for few years.
Fixing a concrete balcony
Fixing a paver pathway
More paint
Continue learning and crafting anything and everything.

Closing thoughts

I really like the fact that this post is only for anyone using RSS. I dont see this as quality content, but more about a slice of my life.

Some of the post ideas I have on my list:

clojure posts
coding posts
house crafting past projects, and current projects
more of this RSS only posts

Not sure if I’ll be able to post again this year, If not, have a happy New Year and see you next year!

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

RSS Exclusive - 2021w50 - Weekly Summary

Fri, 17 Dec 2021 00:00:00 +0000

Pssst - this is a RSS only post. Please keep it secret.
Read more about RSS Club: https://daverupert.com/rss-club/

For a while I’ve been thinking of capturing thoughts / experiences regularly.

You will find thoughts related to my work, my passions, and my life.

Work - Great managers are good partners with their employees

Having a great manager is important. One that will:

create winning conditions
help you grow
support you when needed
let you alone when you dont need anything

I will never work again with (not for) a bad manager.

IT

Learning - OAUTH2 w/ RSA certificate authentication

Been experimenting with RSA based OAUTH token generation.

The process is done this way:

you encode headers with RS256 type
you create json data containing a really short experiration, the necessary audience
base64 url encode both the header and data
your token is the encoded header.data
RSA sign the token and base64 url encode the hash
the signedJwtToken is comprised of base65 url encoded headers.data.signature

During testing, this process is relatively simple, but is tedious.

This can be achieved by implementing pre-scripts in postman, or shell scripts

Some findings:

https://joolfe.github.io/postman-util-lib/
https://stackoverflow.com/questions/49371771/postman-rsa-encryption/59338473#59338473
https://stackoverflow.com/questions/53965446/how-to-sign-a-jwt-with-a-private-key-pem-in-cryptojs

Using jsrsasign-js

var navigator = {};
var window = {};
eval(pm.globals.get("jsrsasign-js"));

function addIAT(request) {
    var iat = Math.floor(Date.now() / 1000) + 257;
    data.iat = iat;
    return data;
}

var header = {"alg" : "RS256","typ" : "JWT"};
var data = {
    "fname": "name",
    "lname": "name",
    "email": "email@domain.com",
    "password": "abc123$"
};

data = addIAT(data);

var privateKey = "-----BEGIN RSA PRIVATE KEY----- \
MIIBOQIBAAJAcrqH0L91/j8sglOeroGyuKr1ABvTkZj0ATLBcvsA91/C7fipAsOn\
RqRPZr4Ja+MCx0Qvdc6JKXa5tSb51bNwxwIDAQABAkBPzI5LE+DuRuKeg6sLlgrJ\
h5+Bw9kUnF6btsH3R78UUANOk0gGlu9yUkYKUkT0SC9c6HDEKpSqILAUsXdx6SOB\
AiEA1FbR++FJ56CEw1BiP7l1drM9Mr1UVvUp8W71IsoZb1MCIQCKUafDLg+vPj1s\
HiEdrPZ3pvzvteXLSuniH15AKHEuPQIhAIsgB519UysMpXBDbtxJ64jGj8Z6/pOr\
NrwV80/EEz45AiBlgTLZ2w2LjuNIWnv26R0eBZ+M0jHGlD06wcZK0uLsCQIgT1kC\
uNcDTERjwEbFKJpXC8zTLSPcaEOlbiriIKMnpNw=\
-----END RSA PRIVATE KEY-----";

var sHeader = JSON.stringify(header);
var sPayload = JSON.stringify(data);

var sJWT = KJUR.jws.JWS.sign(header.alg, sHeader, sPayload, privateKey);

pm.variables.set('token', sJWT);

Emacs - towards stability

project.el

my project.el setup broke. still trying to fix it.

IntelliJ -> Emacs

Continue the IntelliJ -> Emacs research. More details soon.

nixos - home-manager - goimapnotify

My main work involve fixing goimapnotify setup with nix and home-manager.

So far, something is wrong with the config it generates. It seems to be related to single quoting and the password command parameter. When removing the single quotes, my script works.

Configured a network share for our pictures. This will allow us to sort, and organize pictures we’ve been collected for the last 20 years.

The setup involve:

TrueNAS
Samba share
New accounts setup on the NAS
Moving all to that share
Create a 00_inbox folder for things to process

home improvements

This week’s plan is the same as last week. I needed to procrastinate :)

So I’m still going to:

finish some plastering / painting touch-ups.
start planning the construction of my new office.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

RSS Exclusive - 2021w49 - Weekly Summary

Fri, 10 Dec 2021 00:00:00 +0000

Pssst - this is a RSS only post. Please keep it secret.
Read more about RSS Club: https://daverupert.com/rss-club/

For a while I’ve been thinking of capturing thoughts / experiences regularly.

You will find thoughts related to my work, my passions, and my life.

work - new project

Last week I joined a long running project.

Couple of thoughts from this:

Most of challenges are related to organization, and processes
It’s really hard to get in context when no having access to code, tools, information.
- it’s like watching a twitch stream for 8-9 hours per day
- it’s like coding by looking at the key hole
This team is awesome. They are skilled, and easy to work with.
The tools… Locked down, super limited. Not ideal.

work - Lots of industries impose tools to their developers

I’m getting tired of getting all my tools imposed on me. There is a context to it, but it does not make it much easier to digest.

Here I’m talking primarily about my hope to use Linux, and Emacs at work.

This is harder when working for a consulting firm, where there are many layers of decision makers, and it’s a constant beginning every x months.

Object Mother test design pattern using a factory

The Object Mother pattern is great as a way to create your test fixture. Especially for complex domain objects.

I’ve successfully implemented it in the past by combining it with the builder pattern. It made it really nice and fluent.

This week, I’ve seen it implemented as an abstract factory. I think the builder approach works better.

lombok is over appreciated

This library manipulates byte code and inject changes in your Java classes. Some features it brings are related to created accessors, builders, etc…

Couple of things overlooked when including lombok in a tech stack:

# of active contributors (roughly 4)
# of bugs (732 open bugs)
risk of breaking in the future manipulating byte-code is and will break in many major revisions
conflict with other libraries and the JVM runtime in some scenarios, manipulating byte-code can impact dynamic proxies, transactions, AOP matchers, etc.. I experienced some issues in the past between Lombok and H2 DB.
troubleshooting since lombok is byte-code without actual source code, it’s harder to troubleshoot issues.

This is quite a long list of risks for a simple set of features.

There are other alternatives to lombok:

for getters / setters: just use the IDE to generate your accessors
builder: snippets to generate actual builders. some IDEs does it. or get a builder generator that creates java code in target/generated-sources

At this point, you may have guessed I never use lombok unless it’s already in our client’s code.

Changes to my career plan

I’m tired of OO, statically typed languages and tools I need to use at work. (reasons may be explained in later posts)

My career goals are to continue building awesome piece of software using the tools and language I like.

I’m going to focus my learning on Clojure and Javascript/NodeJS to be ready when the time is right.

benlab

This week has been quite active in this area:

setting up mail / gpg with nix home-manager
started to plan how to setup Emacs as a replacement for IntelliJ for Java dev started playing with lsp-mode
Refactoring of the way I deal with my git repositories setup group of repos: private, public, contrib, and others.

my public repos

These are all my public repositories where I’m the only contributor.

New strategy is:

origin will be pointing to sr.ht
mirror will be pointing to github
setup myrepo to push to both remotes
no additional remotes (no upstream)

my private repos

These contains:

project details
stuff not ready to share
private information

New strategy is:

origin points to my own private git server on my nas
no other remotes
checks to make sure remotes dont change or point to public repositories

home improvements

I’ve been renovating my old house for many years. I usually do everything myself.

This week, I’m planning:

finish some plastering / painting touch-ups.
start planning the construction of my new office.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

IntelliJ to Emacs - part 1 - The Plan

Fri, 10 Dec 2021 00:00:00 +0000

A bit of my history with editors and IDEs

I’ve been using Emacs and vim since the mid 90s. First for C/C++, then Java. Java has been my primary language since then.

At that time, I really used Emacs as an unmodified editor, not as a custom tool. Using Makefiles, and search/replace to do most of my work.

Eclipse

Around 2005, I made the switch to Eclipse IDE for my Java coding. Main advantages at the time was navigation, runners, debugger, refactoring and code actions. Eclipse was (and still is) a good IDE for Java coding, but a bad editor.

Some issues:

lots of modal behavior when doing many operations
freezes constantly on such modal operations. You think Emacs freezes, you havent seen Eclipse
Made for the mouse. It has many shortcuts to trigger actions, but any actions that needs user input will require the mouse.
Somewhat hard to extend. Requires to create a whole project, including a build script, generate an jar. All this to do simple things.
configuration hard to maintain
mostly prepackaged solution, that support plugins

IntelliJ

In the past 5-6 years, I’ve been using IntelliJ. Main advantages over Eclipse:

locks less. operations are not modal
a bit more reactive when editing code than Eclipse
better snippets
configuration can be synched on the cloud

Some issues I’ve seen:

2020-2021 was really bad. they made couple of releases that just made things really slow, hanging, or out of sync.
not easier to extend
mostly prepackaged solution, that support plugins
paid solution, while it’s not really a big deal, it makes it more difficult when changing clients, or for new coders on the team. Here it’s more about delays.

eclim, and other similar modes

I tried couple of modes that tried to bring Java to Emacs. Most of the time, basic features were missing, are not supporting new Java versions, or are unstable.

The Ultimate goal: Code Java in Emacs

With lsp-mode, my dream come true: doing all my coding in Emacs.

As mentionned earlier, Eclipse is a good IDE, but not a good editor. With the development made by lsp-mode, I can now hope to bring the IDE features of Eclipse within Emacs.

The Plan

Over the course of next year, I will:

document all the features I use daily in IntelliJ
categorize them
find how to do something similar in Emacs, lsp-mode, or other modes
post regularly my progress

Editing features I’m looking for

Here are the categories I have in mind at this point:

Basic editing features
1. import management
2. code formatting
3. Search/Replace
4. Refactoring
5. Linting
6. code actions
7. Navigation
8. Documentation
Builds
1. Managing dependencies
2. Open dependencies documentation and sources
Runners && Debugger
Integration with frameworks
1. Spring beans
2. spring integration
3. spring data
4. spring initializer
5. Hibernate / JPA
Other supporting tools
1. REST Client
2. Database client
3. JSON/XML editing

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

The last 12 months

Wed, 10 Nov 2021 01:08:08 -0500

Blog silent

I’ve been blog silent for almost a year now.

This blog post is my attempt to capture what I’ve been doing, currently doing and will be doing.

TLDR: most of the last 12 months spent working on my house, working on many client’s projects, coaching, and learning.

Personal

The last 3 years has been a bumpy road in my family. Covid and the restrictions that came with it did not make things simpler.

It made me realize how important my family is to me and that life is not a straight line, we need to adjust constantly.

We all had to adjust to being all together in the house, all day. Kids, wife and me.

Another adjustment, we started home schooling one of my kid mid year. I also started teaching one of my kid how to code.

We found new ways to have fun in family. We’re now enjoying every bit of somewhat normality.

Work and IT

work from home

I work in IT, and obviously worked from home since February 2020. At least I have a job.

The worst of all was during last winter, it was cold, dark, isolated and confined to a repurposed utility room in an office.

I work, and lead teams, meaning I’m often on the phone, helping, coaching. The whole family realized that too due to poor sound proof of the room. I tried many things, but this room is far from ideal.

Due to this reason, I setup a temporary office at my parents place (my old room), upgraded the internet. This actually worked quite well, bringing peace back in the house.

I’ve been adjusting to working remote full time now, and do not look back. I realized how much energy get lost in 15 hours of weekly commute. My employer has announced hybrid work even before Covid restrictions are dropped. They will leave it to the employees to decide if they want to come. They also downsized the office to a big extend already, so it’s there to stay.

what’s coming: sustainability, coaching and helping others

the IT world has been in unsustainable expansion in 2020-2021. So much demand, that “experienced” coders are less experienced than ever. Just recently, I got a proposal to move away from client projects, and help, coach less experienced coders using a mix of discussions, internal projects that brings value to the organization. This is not a “no tech” title, I code more than ever. This is still quite new, possibly more to come in future blog posts.

IT / community

I’ve meet a great team of different/like minded community built by David Wilson at System Crafters. Made me learn stuff, help others, and rediscover IRC. I also discovered awesome people like David, Kartinks, Acdw, Fade, AlphaPapa, TitanOfOld, and many more.

Home

As earlier introduced, a lot of my (free) time has been spent around the house, so what else can I do other than starting/finishing house projects I planned to do, but never had the time or energy to start or complete them.

Here is a list of projects I’ve done over the last 8 months:

sanding / refinishing oak floors
plastering, painting walls
new trims and baseboards
installed new doors
new tiles in the house entrance
pool deck
stairs and rails in the backyard exit
concrete repair

My next project is to build a small office, soundproofed obviously. Construction started last week.

Future posts?

here are some of the subjects I have plans for future blog posts:

Moving from IntelliJ to Emacs/LSP for java development
Various coding posts
Home Crafter’s version of: The Office
New posts around my evolving work setup
What ever else happens in my life that I want to talk about

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

RSS Exclusive - Proof it's Always a Good Time to Take Vacation

Thu, 24 Dec 2020 10:18:14 -0500

Pssst - this is a RSS only post. Please keep it secret.
Read more about RSS Club: https://daverupert.com/rss-club/

No matter what managers or you think

No matter what managers or you think, work is never a good reason to wait for your vacation!

And I can prove it!

The proof

Lets start with some facts:

vacation is good for you, for your mental health
vacation is good for your employer, a well rested person is more likely to be motivated, and more productive
we are smart and can plan some time ahead
In many places, you loose rights to vacation each year, sometimes above a threshold. Yeah, they may pay you instead. But at one point, money has less benefit than facts 1 and 2

Impediments to take your vacation:

you. sometimes we are our own impediment to take vacation
in some jobs, it’s seems like it’s never a good time to take vacation. There is always something coming up
Some managers constantly ask you to move your vacation, or skip it

Now the proof:

managers that ask you to constantly make compromise only think about short term $$, has poor planning ability, and does not think about long term sustainability. I certainly would not trust that manager.
If it’s never a good time to take vacation, therefore it’s always a good time to take your vacation
if you always think of staying because “what will they do without me”, see vacation as an investment in you and your family. If you work can’t survive without you, maybe this is a hint around planning or too low bus factor.

For all these reasons, if you plan a bit ahead of time, vacation is “FYI”, not “can I”.

I would not trust a company or manager that ask me to make unreasonable compromise to my vacation, or ask my colleagues the same. It would be enough for me to seek for a new job.

I obviously apply this based on my own experience in IT and R&D jobs, but I strongly believe it applies to all fields with some tweaks (on some jobs, you cant all go on vacation at the same time, union jobs have special rules).

I’ve seen friends suffering from this situation.

I’m in no way in this situation, actually quite the opposite, he’s asking us to take time off.

I thought about making this post on my first day of vacation, thinking maybe others may make unreasonable compromise.

Happy and safe Holidays!

This is day 18 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

Lets see if I can make it to 20 before year end :D

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Lean UX, large code

Wed, 23 Dec 2020 20:49:46 -0500

It all started that day

Two weekends ago, bored, I was looking for something to do.

I remembered posts on fosstodon about some website size ranking called “The 512kb club”.

Last time I worked on my blog setup, I tried porting it to Emacs org publish.

Following the instructions on 512KB site, I ran my site through GTMetrix.

To my surprise, I discovered my simple blog index page was roughly 600KB.

What!!! 600KB, that is 600000 characters, just for an index page that has a listing of my 15 or so blog posts. 600KB for 15 posts, this is an average of 40KB per titles. Even more insane!

The whole point of the 512KB is the fight bloat, promote real lean website. My blog was far from being lean.

I was motivated, I had a plan: make my blog leaner

What is the root cause

I never optimize without data, so a quick look at GTMetrix told me the problem: the CSS, and fonts were the majority of that 600KB.

I got these from the theme I imported during my initial setup (on my 1st post). It was based of a Hugo theme posted on GitHub made to look like motherfuckingwebsite, but I had to admit, like many site with lean UX, under the hood it was not lean.

In search of a real lean CSS

I could have just modified the CSS, but with that much dead code, there is just no point of doing so.

I’m also a practical coder, I’m not the only one with the idea of a lean CSS. After couple of searches, I found new.css, a CSS that brings lean UX in a relatively small package (4.5KB).

The best way to succeed would be to generate this site with plain semantic HTML, trying to avoid DIVs and other general container. Possibly sticking to HTML5.

The wrong approach

I decided to continue porting my blog to Emacs org publish. The idea being it would remove Hugo from the equation since I already use Emacs org-mode for this blog.

After couple of hours of work, I finally succeeded: my blog was rendering like before, with the same CSS as before, and I even got the RSS properly rendered.

Next would be the CSS, so I decided to open up the produced HTML to get ready for some CSS work. The result? I was way further from my original goal. The generated page was just full of divs, almost unreadable.

I tried to fix this, but the more I fixed issues in the generator (I had to customize many Emacs functions in lisp), the further I was to keep things simple.

I just had to admit, that the only thing productive about this effort was the learning I gained :P

Second try

I had to find a simpler solution. Could I make this work with my Hugo setup? I discovered my CSS was bloat, but the generated HTML was actually really good and lean.

30 minutes is all it took to replace the existing theme with a simpler one that imports new.css + the font suggested in the new.css site.

Satisfied with the look, the simpler CSS, I decided to push it to my server.

New data point

Time for a new size check. Result: 8.5KB

Success!!!

Time to submit my site to 512kb.club

Satisfied with this new improvement, I submitted my site to 512kb club, happy to be somewhere between 13 and 15 position (many people are submitting sites).

Can I do better?

What can I do to improve?

I could always try to reduce that 4.5KB CSS, but this would mean spending time learning this code, or writing my own. Not enough time for that yet.

What else can I learn from the GTMetrix site? ohh, a 404 on favicon.ico cost roughly 1KB

What If I could create the smallest possible file, to prevent my 404 page to render. This could work.

I did not know that you can create a favicon inlined in the HEAD section of a page.

<link href="data:image/x-icon;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQEAYAAABPYyMiAAAABmJLR0T///////8JWPfcAAAACXBIWXMAAABIAAAASABGyWs+AAAAF0lEQVRIx2NgGAWjYBSMglEwCkbBSAcACBAAAeaR9cIAAAAASUVORK5CYII=" rel="icon" type="image/x-icon" />

his is all you need to generate a small blank favicon.

Metrics after this improvement? 8KB.

his is 75 times smaller than the original size. This is 0.53 KB per post instead of ~40 KB per post.

At the time of writing this post, this puts me at the top 12 smallest sites on the 512kb club.

Before / After

Figure 1: Before

Figure 2: After

Thoughts

Creating simple solutions is not necessarily easy, but is fun and rewarding.

At the same time, you can do a lot with not much time.

I wish to inspire peers (especially in the IT industry). The web is bloat, but it’s not better in backend code.

Note: as a side note, I still motivate myself with 100DaysToOffload. It will take some time to reach my goal (17 posts so far), but I will get there sometime in the next few months, or years.

This is day 17 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Chezmoi Merging

Sat, 03 Oct 2020 13:13:12 -0400

Adjusting chezmoi to my workflow

Today, I’ve been adjusting chezmoi to match my workflow for editing dotfiles.

You may be interested to read my previous post where I go over how to setup and use chezmoi: Dotfiles management using chezmoi - How I Use Linux Desktop at Work Part5

The workflow I’m seeking:

edit the files in place the same way I edit any files. I don’t want to use chezmoi edit on some files, and open files directly in vim or Emacs for other files.
run chezmoi merge on all files changed
configure chezmoi to automatically commit and push

Right now, I need to remember not to run chezmoi apply to prevent loosing changes.

Issues with default chezmoi behavior

Implementing my workflow is not easily supported by chezmoi:

chezmoi does not support changing the order of files specified when calling the mergetool. kdiff3 merge tool requires a different order to setup base, local, and remote files.
there are no chezmoi merge –all commands to merge all files that changed

implementing the workflow

setting up kdiff3 as my mergetool

chezmoi has an option to specify the mergetool:

1
2
3

[merge]
  command = kdiff3
  args = [ "-m" ]

This has the first issue listed above: kdiff3 is called with files in the wrong order

Here is my step process to solve this problem

step 1: lets create our own kdiff wrapper script

I could have contributed a change to chezmoi to get this working (and I may will), but I need a solution now.

So I choose to create a script in ~/.local/bin/ called kdiff3merge:

#!/bin/sh
local="$1"
chezmoi="$2"
base="$3"

kdiff3 -m "$base" "$chezmoi" "$local" -o "$chezmoi"

and modify my ~/.config/chezomi/chezmoi.toml:

1
2

[merge]
  command = "kdiff3merge"

note: ~/.local/bin is in my path

Now, when calling chezmoi merge ~/.zshenv I get a good result:

Now I just accept the automatically merged changes by saving the changes and closing kdiff3.

One issue I’ve found with this is it creates a dot_zshenv.orig file corresponding to the chezmoi controlled file before the change was applied. I don’t need these since I use version control.

#!/bin/sh
local="$1"
chezmoi="$2"
base="$3"

kdiff3 -m "$base" "$chezmoi" "$local" -o "$chezmoi"

test -f "$chezmoi.orig" && rm -f "$chezmoi.orig"

Now orig files are gone.

This is what I needed to fix the first chezmoi limitation I found.

Run chezmoi merge on all files changed

Chezmoi has couple of commands that gives information about changes:

chezmoi diff
chezmoi apply -n -v

Both commands are actually too verbose. I could not find a way to just print the list of files to bring in.

Obviously, I ended up creating an alias that does this:

`1`	`alias cmm="chezmoi diff -f git \| grep -- ^--- \| sed -e 's/^--- a\///' \| xargs -r chezmoi merge"`

If we deconstruct this command:

chezmoi diff -f git: prints all changed files, with their changes
grep – ^—: this one filter in lines which starts with —. the – prefix is to tell grep the — are not part of the “-” options (like -V)
sed -e ’s/^— a\///’: this command remove from lines found on step 2, the “— a/” prefix found. at this point, we end up with file path/names only
xargs -r chezmoi merge: runs chezmoi merge on all files found, or run nothing if no files found

Problem solved. I regularly run the cmm alias to bring my changes in chezmoi.

Commit and push my changes

chezmoi supports that off the shelf and can be enabled with the chezmoi.toml:

[sourceVCS]
  autoCommit = true
  autoPush = true

[merge]
  command = "kdiff3merge"

Conclusion and thoughts

CLI tools offer quick ways to extend or customize to fit your habits.

That being said, my scripts may break at any chezmoi release update.

I just cloned the chezmoi repo, and looking to contribute more permanent changes to chezmoi diff and chezmoi merge commands. This will be a good opportunity to learn go.

This is day 16 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Home data storage

Sat, 05 Sep 2020 20:26:48 -0400

A need to own your data

The workday today ended with my friend and work buddy telling me it’s about time he own and centralize his data (for him, wife and kids).

I’ve had servers since the 90s, so obviously I had solutions for him :P

Like mine, he and his family use a variety of devices at home, phones, laptop, desktop.

The two obvious choice to self host your data are:

a Network-Attached Storage (NAS) server
a decentralized network of computers replicating your data (aka: Syncthing)

While Syncthing is great for smaller data sets, it requires to share the same data set at least on 2 machines. When you have terabytes of data, it becomes not practical.

So this post is about running your NAS. A NAS offers multiple ways to access your data like NFS, Samba, SSH/SFTP, WebDav and more. How to setup your NAS is beyond this post, and will possibly be the topic of a future post.

Made for you or home made?

There are two main track for running your own NAS at home:

Getting a store bought NAS appliance (synology, Qnap, Netgear, and others)
Building your own

Store bought NAS

Here are some reasons to choose a store bought NAS appliance:

Simplicity, beside disks, you get the box, motherboard, CPU, ram, SATA controller, and software sorted for you. Just pick a brand and the number of drives supported, and you’re done with choices
Fast to get started, put drives in, boot and setup
Usually small footprint

Some minuses:

Usually expensive, especially if you want to put more than 2 drives
Not flexible, software is usually proprietary and the computer is lower power unless you pay big $$$
Data at risk when the hardware dies. You may want to setup redundancy which on these NAS is implemented by proprietary setup. If the NAS dies, you may have to find the same branch, possibly the same model/family to recover your data.

Build your own

Some pros:

Make your own, you make all the choices
Usually cheaper for higher capacity NAS than a store bought appliance
easier to put back online if one of the key components dies since it will be based on software RAID
more software options including software RAID
For Geeks, more fun :D

Obvious cons:

more decisions to make
more complex
Willing to learn, read, research, break things, especially if you have limited knowledge of running servers. If you want stuff done for you, building your own may not be the right fit.

My family would find it obvious that I suggested my friend to build his own :D

Some things you should plan for:

a big box, with enough room for your drives and at least 1/2 more space to add drives as your needs change.
OS Drive. You usually want to store the OS on a separate disk. I really small SSD is a good choice.
disk controller
1. onboard: already included. slower, less quality, and usually more limited in number of ports. rarely more than 4/6 SATA ports
2. SAS/SATA card: much faster, robust. you can find cards used in server cheap on eBay. A 8 ports LSI SAS 2008 card cost less than 100$.
Storage Drives. Two paths SAS or SATA. You may want to vary brand or batch to reduce the risk for multiple disk to fail
1. SAS, robust, but expensive. you can buy used drives though
2. SATA: easily obtainable used
a decent CPU/MB/Memory, possibly with ECC Ram. You don’t need to buy new, but decent hardware. Some people run out of raspberry pi, but this limits performance due to requiring USB drives which are slower (USB) than SATA/SAS drives on a SAS or SATA bus. Make sure you have PCIe expansion to host

My setup:

Cheap Rosewill 4U server box
i3 / supermicro / 16 gig ECC
many 3T WD RED drives bought at different time
a small 128 gig SSD
a controller LSi 9210-8i running a P20 IT pass-through firmware needed to run FreeNAS / ZFS and bypass the hardware RAID on the LSI card.

Redundant array of inexpensive drives (RAID)

Lets start with some definitions.

RAID is a way to describes different arrays of drives to achieve increased capacity, or redundancy, or both.

Here are some important ones:

RAID 0: a stripe of 1 or more drives. You basically have multiple drives acting as one big drive (the sum of all drives). This adds capacity to make a bigger “logical” drive
RAID 1: 2 disk or more setup as mirrors. 2 drives is the minimum, but some systems allow to add more drives to the mirror. The obvious advantage is redundancy, a lesser known is faster read (you have more disks to read from); The main disadvantages, cost and write has to be done on multiple drives.
RAID 5 or RAIDZ (ZFS): Act as a stripe, but address redundancy by adding one drive to allow a parity bit. Don’t be mistaken, the additional parity bits are not stored on one single drive, but spread on all drives. ZFS support additional parity bits for added redundancy. Some pros: capacity, lower cost than a mirror. Cons: Slower read than a RAID 1.
RAID 10: combine RAID 1 (mirror) to RAID 0 (stripe). In this configuration, mirrors are concatenated into a stripe for added capacity. This combines both redundancy, capacity and read performance, but increase cost.

My favorite RAID, RAID 10. Cost / Gig is so low these days that it’s less of a problem these days.

Volume Manager + File systems

I’ve been playing with two main setup:

Linux Volume Manager (LVM) to give me software RAID, and EXT3/4 for file system
ZFS for both volume manager and file system

I won’t into the details of each, both are great and my favorite is ZFS. It’s not more complex than LVM / EXT3/4 and it gives additional features like snapshots, and easy replication.

Software

Here you have plenty of options:

your own custom Linux + other software
OpenMediaVault (OMV), a linux based NAS OS
FreeNAS, a FreeBSD based NAS OS
Other proprietary NAS software

I’ve tried my own, OMV and FreeNAS. With ZFS support, solid feature set, and stability, FreeNAS is my go to OS for my NAS.

To install it, you have to boot the installation image, install the OS on your OS drive, then configure your ZFS pool and data volumes.

Backup

When you plan for a NAS you must also consider backup:

local
remote

You need both local and remote backups to be resilient to lightning, flood, someone stealing, or just your own mistakes.

Local backup

For local backup, you can setup ZFS snapshots. ZFS snapshots are taking a “picture” of your files at a given time. Some of my ZFS volumes have daily snapshots, some have hourly snapshots and I usually keep them for one month.

ZFS Snapshot + RAID 10 gives me a relatively fast recovery time in case of issues or failures.

Remote backup

You can either buy hosting for your backup, which is relatively slow, could be costly, and could have transfer limits. If you have of data to backup, both speed and possibly cost will become an issue.

The other option is to host a low power machine with enough drives to get your data. I currently have an old core 2 duo running at my parents place. I run daily ZFS snapshot replication which copies my snapshots to the remote server every day. The worst I can loose is one day of data.

Now my friend needs a remote backup solution, so we both agreed to host each other’s backup. For this, he’ll send me a large drive, and I’ll send him one too (and decommission my old server at my parents place). Since I want to keep my data to myself, I will most probably use Borg backup to backup and encrypt my latest ZFS snapshot, and send this over to my friend’s NAS.

You also may have a friend or family member who can host an old server, and maybe share the cost.

Other services

Once your NAS is setup, you can add other distribution services, like NextCloud, plex/emby, airsonic (music), or calibre (books). You can run all of these on FreeNAS if you have enough RAM/CPU.

Thought

Your NAS does not have to be that powerful, if all you have is documents and some pictures, a raspberry pi or similar is enough. If your NAS has multiple users and server lots of (large) files, it may require a bigger setup like the one describes in this post.

Setting up your own NAS is possibly the first step to own your data.

This is day 15 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Netboot Pxe

Thu, 03 Sep 2020 20:46:47 -0400

Where are my USB drives?

Have you been there too?

I seems to never find my spare USB drives when I need them. This week, I wanted to re-install my laptop but could not find an unused USB drive to put the ISO on.

This is when I found PXE and NetBoot.xyz image.

What is PXE?

PXE stands for Preboot eXecution Environment. It’s a way to distribute boot images over network.

It uses a combination of TFTP (Tiny File Transfer Protocol) for transferring images and DHCP for discovery.

It enables to boot most machines over network.

netboot.xyz

While searching how to setup PXE, I’ve discovered netboot.xyz. If you boot one of their images, you will get a menu which allows you to select live images, install images, rescue images.

Here is what it looks like:

And selecting network installs shows this:

Select an image, it will download and boot it.

TFTP server

Self hosting TFTP is relatively easy if you have a WRT router, PfSense/OpenSense, FreeNAS. It’s not much more difficult to install on any Linux server.

In my case, I have the choice between:

DD-WRT :: not the best since it’s hiding behind my network in a NAT network.
OpnSense firewall :: Would work but has limited space
other servers :: more complex than other solutions
FreeNAS :: ended up choosing my NAS, where it’s primary role is to serve files.

The setup is straight forward:

create a folder where you can store the netboot images
Enable the TFTP service
configure TFTP to use the folder created as it’s main directory
store the netboot.xyz images: netboot.xyz.kpxe for regular boot, and netboot.xyz.efi for EFI booting.

If you want to verify your setup, just download a TFTP client (for example, atftp), connect to the TFTP server and get netboot.xyz.kpxe. This should download the images served by your TFTP server.

DHCP

Once you have TFTP working, the last piece missing is the DHCP configuration.

What happens is your DHCP server, which most PC get an IP from, can give additional information, like the TFTP / images to use during PXE network booting.

This seems to be limited to some routers, but most FOSS firewall or routers allow you to configure PXE.

On my OpnSense Firewall, I need to:

enable network booting
set the TFTP server to my FreeNAS server IP
define the default image with netboot.xyz.kpxe and the EFI 32/64 with netboot.xyz.efi

Booting up

Now, to boot using PXE on any PC or even in VirtualBox, you need to select network in the boot menu. After getting an IP address, you should boot the netboot.xyz image.

Thoughts

I’ve missed writing on my blog… Life, vacation, making major backyard refactor put me away from the keyboard for some time.

Special congrats to folks who made it to the 100 posts. This is quite an achievement.

I don’t think I’ll get this anytime soon, but I plan to continue posting regularly.

This is day 14 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Working From Home

Wed, 02 Sep 2020 00:17:14 -0400

A curse and/or a blessing?

Like many of us, I’ve been exclusively working from home since March.

I’ve read couple of post on fosstodon mentioning how much people want to go back to the office. I have a totally different experience.

Adjustments

Lets start with the obstacles I’ve experienced and some solutions that works for me:

Doing long hours: I start later in the morning so that I’m done by 5 without too much overtime.
Work is always around, in your mind: I hide my work gear / laptop after work so that I don’t see it
Feel alone, disconnected from colleagues: We hang out regularly on a audio/video team chat, people come and go at will
Distracting my family, I was working in our family room. Not so great idea when I spend a good time of the day on calls.: I repurposed a room into an office
Motivation: This one was adjusted a while ago, have some routine in the morning, breakfast, dress up, go outside for couple of minutes, then go in my office. This prepare my mind to start working.

Some people also have to take care of young kids, that could definitively be problematic. I’m not in this situation, my kids are teenagers.

What makes it so great for me

Here are some benefits from working from home:

Overall, my stress is lower since I started.
My commute went from 15 hours a week down to nothing.
I spend more time with my family
I have a better setup at home (screen, chair, desk)
I take my lunches outside in my backyard
I’m not stressed of missing my train if a meeting last 5 minutes longer than it should
I can listen to music without my headphones
Less unnecessary distractions from colleagues
Not an open space. I do like team rooms, but I hate rooms that can sit 100 people

Thoughts

Your experience could be totally different, this is really a personal thing.

Once Covid restrictions are lifted, I would gladly meet my teammates at the office once in a while, but regular attendance to the office is no longer part of my life.

Working from home is a perfect fit for me :D

This is day 13 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Ditch WYSIWYG apps and switch to plaintext tools

Fri, 31 Jul 2020 00:00:00 +0000

On this post, I’ll go over WYSIWYG apps, and effective alternatives.

Like it or not, in tech jobs, we have to write documentation and create diagrams to share our ideas and solutions.

The traditional approach is to use WYSIWYG (what you see is what you get) apps. The approach with WYSIWYG apps is to allow you to edit the graphical representation of your document. This is now mainstream in spreadsheet apps, word processors, presentation apps, and diagram tools. Using MS tools, it’s word, excel, PowerPoint, vision, or the equivalent LibreOffice writer, calc, impress. These apps use more or less open formats, at the same time have you ever tried to understand one? These formats are made for software, not for humans.

The older of us may remember at a time, WYSIWYG views were non existent and then became an addon feature (Anyone remembers Lotus 1-2-3?). We had the sense of progress!!! We still had stuff like sgml documents around, but for a while, plaintext documents almost disappeared.

Anti-productivity

Portability

Going from one format to another is kind of difficult.

Collaboration

Collaborating on one ppt or docx document is kind of a nightmare. Same goes with LibreOffice equivalent.

This one evolved over time, we now have web applications to do this. Even if this is working, we end up with centralized services that cannot work offline.

Version control

Difficult to implement of a complex file format with graphical tools.

Structure and data mixed with presentation

Reason why we have CSS and HTML, separation of concerns. For diagram tools, you also have physical position to handle as well.

Have you ever tried to build a diagram live during a session with a client? This may prove to be quite stressful and unproductive. The “oh, can you add a box there, shift this lower, can you make all the boxes red, etc…” are becoming quite complex to perform.

There is also maintenance of these documents, not too difficult when we just created them, but quite painful.

Alternatives

Plaintext is one of the best alternative to what we are used to. We can share, read, modify with our text editor, version control in git, collaborate, easily write/modify them during live sessions.

I’ve been transitioning to plaintext format for couple of years now, use git to share and control changes.

Here are some known and less known alternatives to WYSIWYG apps:

work processors

This is the easiest of the bunch, many of us probably use an alternative to word or writer:

markdown :: possibly the most popular of the gang. quite good for simple stuff. No specifications means 100 variants. Good for READMEs, short docs.
commonmark :: a tentative to create a markdown specification. Not sure how successful this is. Use in same context as markdown
asciidoc :: a simpler markup facade to docbook, a full fledged document standard created by oasis. This one is quite good for writing more complex documents. It both replaces word/writer and is an alternative to LaTex.
org-mode :: mostly Emacs specific, can be used as both a markdown replacement, as well as an outliner / task list / note taking app.

Honorable mentions:

LaTex :: I never really used it, but some love it and it has similar properties than other plaintext formats. The showstopper for me is the higher barrier of entry / learning curve.
reStructuredText :: seems similar than others. Haven’t tried because it is similar to other more popular formats.

Spreadsheet

This area is still dominated by Excel and Calc. I usually take CVS data feeds which address part of the issues with Excel / Calc. This is an areas I haven’t experimented too much.

Honorable mentions:

org-mode :: offers simple aggregation functions for simple spreadsheet in org tables.
gnuplot :: can be used to replace graphs
sc :: ncurses TUI spreadsheet. This one feels like Lotus 1-2-3 :). It is now discontinued. I barely used it, but it seemed fined.

Presentation

Like word processors, many of us use plaintext alternatives.

The one I’ve used

org-reveal :: org-mode + reveal.js. my today’s favorite setup
org-mode w/ LaTex Beamer :: when I need to share my org-mode presentations

Honorable mentions:

LaTex Beamer :: often seen in scholar’s presentations.
Sent :: extra minimalist presentation software inspired by the Takahashi technique

Diagrams

These tools are ones I use almost daily. I work with relatively large teams, and have to share ideas with coders, business analyst, clients, and many more.

Some people are listeners, some are visual. I then to create diagrams that has both text and visual elements.

How does it work for diagrams?

You “code” your diagram like here in this flowchart:

@startuml
Alice -> Bob: Authentication Request
Bob --> Alice: Authentication Response

Alice -> Bob: Another authentication Request
Alice <-- Bob: Another authentication Response
@enduml

Here are some of the tools I use:

plantuml :: my goto tool for diagrams, don’t be fooled by the UML in it’s name, it’s not like most UML tools that enforce strict UML rules. In many ways, it’s main role is to put boxes with text/icons in a graph and link them when related. In follow-up posts, I will cover some of the diagrams you can create.
mermaid :: Similar to plantuml, but in JavaScript. I’ve used mermaid-cli to create git branching diagrams
graphviz :: when the other 2 tools don’t give want I want. Really flexible. Plantuml uses graphviz as a backend.
nwdiag :: build network diagrams

Honorable mentions:

Ditaa :: draw ASCII boxes and it renders them in an image
rackdiag/nwdiag :: build a server rack diagram
erd :: entity relation diagrams
imagemagik convert / identify :: add text to images
banner / cowsay :: generate plain text art and banners
shaape :: ascii art to image converter
syntrax :: railroad diagrams. could be used for branching diagrams. next one on my list

Most of these diagrams can be embedded in markdown, asciidoc and org-mode files including presentations.

Some references for diagram tools:

Thoughts

I just could barely touch the subject but this is already too long. I hope I could give you some ideas and alternatives to tools most people use.

While doing this post, I discovered some cool tools I did not know. This may be useful for some of my work related documents. You may see more post around this in the future.

This is day 12 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Learning Gentoo

Sat, 25 Jul 2020 00:00:00 +0000

Been quite busy in the last couple of weeks. What kept me busy? Family being on vacation, my backyard landscaping project, and wiping Arch from my laptop with Gentoo….

I had no clue what was really Gentoo before wiping my laptop clean. Like with any major learning, the easiest way to learn is to just dive in and practice until you understand enough.

This is a learning experiment, so I’m not sure if I will keep using Gentoo after this.

I can tell that so far, I’ve achieved want I wanted: Being productive on my laptop again. Since I’m using this laptop to write this blog, you can imagine that this put a pause to my posts :-).

This post will be about what I learned, and also about the learning journey. There are plenty of quality guides, references, videos on Gentoo, I’m not going to repeat too much there.

My OS experience

On the OS front, I started with dos, win 3, win95, then switched to Slackware, Debian, Ubuntu and Arch. I still actively use Debian (server), Ubuntu at work, and Arch on my home desktop/laptop.

I sysadmin my lab, also code and deploy to Linux and Unix since I started using Linux.

In the recent years, I’ve been transitioning my desktop to Arch to get more recent software, and I run Debian on my servers.

Why Gentoo?

Lets start with my background: I started coding when I was 8yo and I’m now 44. I’ve worked on so many things, languages, OSes. I experimented, broke many things, fixed and learned so many things and I still keep learning. I do think that anyone successful in this field should have curiosity, courage to try new things, and constant habit to self improve things. If you stopped learning, you probably don’t do enough.

Gentoo has a lot to give for someone who likes to understand and thinker with things. So I think it may be the perfect match for me.

In a nutshell, I want to learn something different that I can shape to my liking.

Gentoo, the beginning

About 1 month ago, I tried to learn Gentoo without much success. In this case, I just installed it in a VM. What was wrong? commitment! I was doing nothing useful on the VM, so I lost track and beside the installation, I did not learn much.

How to fix commitment?

Lets wipe my current working setup, and get started with Gentoo!!!

Booting, 3 takes

I believe booting was the most difficult part of the whole installation. Some reasons of failures:

Was following the non-uefi boot setup while my MB was setup for UEFI
This is a laptop, so needed LVM with dm-crypt. This required some boot options. Could not find info about this in the handbook, but I then found this really good guide on setting up LVM, dm-crypt on Gentoo: https://wiki.gentoo.org/wiki/Full%5FDisk%5FEncryption%5FFrom%5FScratch%5FSimplified. Things worked after that.

The good thing about it, I practiced base install 3 times :P

Some random learning

Here are some random learning on Gentoo

The Gentoo handbook is quite great. The wiki too, just a tiny bit behind the ArchWiki.
Large software or ones with many dependencies increase compile time. This forced me to re-question the software I use
You may not want to compile Firefox yourself
Start with no to minimal number of USE flags. Set use flags to specific packages unless it does cover a large set of packages. For example, X, NetworkManager, PulseAudio could be set system wide, but JPEG could be set to ffmpeg only.
after getting used to portage configuration files, flaggie is a quite nice tool to simplify this process
List of use flags with description that a package support can be listed using equery u <package name>
some packages have an insane amount of dependencies. Try to find alternatives before
doas is simple and nice, but sudo handles better password request
openrc is working well, except for networking. Could not get network to be properly switching between LAN and WiFi. I ended up using NetworkManager with nmcli/nmtui
Don’t forget to emerge –change-use @world after changes to global USE flags.
Browsers are not that fun to compile. One night for Qutebrowser and it’s dependencies on a 4 thread basic CPU.
Always use the –ask when emerging packages.
Config files temp files are called ._cfgsomething, I usually do a diff between this new file and the original file to spot changes, and just overwrite the original file with the new one.
Most of the tools / software I use are fast to compile except:
1. the kernel. I need to configure my own instead of using genkernel.
2. ripgrep, took ages due to compiling rust.
3. kdiff3, I actually C-c the process since it depends on the world. I plan on using ediff instead.
4. mpv is complex to compile, it depends on ffmpeg that has pages long of use flags and other dependencies
It’s easy to just set the use flags I understand and update them later if needed. Much better than trying to get everything perfect the first time.
Gentoo is not that difficult when you dive in and have prior Linux experience. I started the process last Friday, and I took 5 days to get back a functional setup.
Unclear if this is just real or not, but my laptop feels faster

Thoughts

Was it worth it? Hell yeah!

Will I keep using this on my personal laptop? Most probably, unless it comes to a point of always breaking (which is totally not the case). Also, I cannot justify spending another couple of days yet to go back to Arch when I can do everything on Gentoo.

Will I use this at work? Most probably not, hard to justify the compile time, but at home, it does not really matter. Ubuntu or Arch are better options.

Now I’m back with a working blogging setup, you can expect more posts in the next few days.

This is day 11 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Patching Tools - St

Sun, 12 Jul 2020 00:00:00 +0000

As part of my renewed dotfiles setup and dev box setup, I’m taking the opportunity to update my core suckless.org tools: st, dwm, dmenu.

I’ve been using these for some time, and it’s great for me to be able to customize tools I use.

In this post, I’ll re-create a new ST fork using up to date code base, and new patches that will work for me.

Note: The focus will be on branching strategy and resolving merge conflicts. The aim is to show an effective way to leverage git and merge tools to integrate patches. You may want to read my previous post describing how merge conflicts can be resolved using tools.

initial setup

As you may know, git is a distributed version control system that can sync with multiple remotes.

To get started, I usually:

create my own git repository
clone this repository locally (origin)
define an upstream remote
merge the upstream/master branch into my local master branch
update my remote origin

Lets do this:

git clone git@github.com:benoitj/st.git
cd st
git remote add upstream https://git.suckless.org/st
git fetch upstream
git merge upstream/master
git push

The resulting remote setup is:

`1`	`git remote -v`

and my current branch setup:

`1`	`git branch -a -vv`

You can see that my local master branch tracks my github’s origin/master branch.

Patching strategy

For each patch, I will:

create a branch from the commit the patch was created from.
apply the patch. We should not get conflicts here.
merge the patch to master branch (may need conflict resolution. Read my previous post)

How do we know how to create the branch?

In the case of most patches, the filename indicate some context. In the following example, we will apply the patch st-xresources-20200604-9ba7ecf.diff. From the file name, we understand the patch was created June 4th from the commit hash 9ba7ecf.

Now the trick is to create our branch from that commit. Reading the git documentation, you can learn this is done with the syntax: git branch [commit hash]

Sometimes the commit hash does not match the history, in this case, use git log and find a commit hash close to the patch creation date.

Patching ST

Lets do this and apply the patch:

mkdir patches
wget -P patches https://st.suckless.org/patches/xresources/st-xresources-20200604-9ba7ecf.diff
git branch patches/xresources 9ba7ecf
git checkout patches/xresources
git apply < patches/st-xresources-20200604-9ba7ecf.diff

And we can see what has changed:

`1`	`git diff`

and lets confirm it worked:

1
2

make clean all
./st

Everything’s good, now is a good time to commit our changes:

1
2

git add x.c config.def.h patches
git commit -m "applying patch st-xresources-20200604-9ba7ecf.diff"

And push:

`1`	`git push --set-upstream origin patches/xresources`

and merge the branch to master:

1
2

git checkout master
git merge patches/xresources

I will do the same for the following patches: externalpipe and scrollback:

wget -P patches https://st.suckless.org/patches/scrollback/st-scrollback-20200419-72e3f6c.diff
git branch patches/scrollback 72e3f6c
git checkout patches/scrollback
git apply < patches/st-scrollback-20200419-72e3f6c.diff
make clean all

1
2
3

git add st.c st.h config.def.h patches
git commit -m "applying patch st-scrollback-20200419-72e3f6c.diff"
git push --set-upstream origin patches/scrollback

1
2

git checkout master
git merge patches/scrollback

If I build master, I have both scrollback and xresources patches included:

1
2

rm config.h
make clean all

Merge conflicts

You may get couple of merge conflicts around keybindings, configuration, and similar.

They are usually really simple to solve using a merge tool as described in this previous post.

I had no conflict patching st, but had some when patching dmenu. Here are some examples:

Example of applying borderoption patch on top of border patch

git checkout master
git merge patches/border
git merge patches/borderoption
git mergetool
git commit

Example of such conflict when merging border and borderoption on dmenu master branch:

In this case, I will select the remote (the borderoption branch) since it makes the attribute mutable.

Example of applying the dmenu fuzzymatch patch on top of center patch

git checkout master
git merge patches/center
git merge patches/fuzzymatch
git mergetool
git commit

In this case, we need to take both the local (center patch) and the remote (fuzzymatch patch), so I select C and B to get this result:

Thoughts and what’s next

I hope I was able to show how simpler patching tools can be when using a branching strategy, branching from the right version, and using tools to resolve conflicts.

Yes you can do all this manually on the same branch, but it’s takes more time and it is much more error prone. Tools are better than us to identify patterns and repeat tasks.

Have fun patching! :)

This is day 10 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Git Merge Conflicts - Git patching and resolving conflict (part 2)

Sun, 28 Jun 2020 00:00:00 -0400

I’m creating this post after seeing many people suffering and manually addressing merge conflicts while integrating patches with dwm, st, or dmenu code base.

The most common issues seen:

applying patches on latest from master
using patch instead of git to patch
resolving conflict manually

In my previous git post, I went over how to create and apply patches to a git repository.

In this post, I’ll show how conflict can be solved using tools.

Note: I’m going to use git, but these techniques are not unique to git.

Basic setup

setup git merge tool

In this example, I’ll start by using meld diff tool. In a later section, I’ll also show different tools.

You first need to install meld.

It can be installed on Linux, windows, OS X. Most distribution package it, or head over to their website.

After it’s installed, we need to tell git to use this tool as a mergetool:

`1`	`git config --global --replace-all merge.tool meld`

You can then inspect the your ~/.gitconfig and you should see a section like this:

1
2

[merge]
        tool = meld

create a local repository for the experiment

To do some experiment, I’m creating a local git repository, with 1 file with some text:

mkdir merge-conflicts-test
cd merge-conflicts-test
git init
cat > file1 <<EOT
original line 1
original line 2
original line 3
EOT
git add file1
git commit -m "initial commit"

We have a new repository with file1 containing text:

1
2
3

original line 1
original line 2
original line 3

Lets inspect the history by running the command:

`1`	`git log --oneline --decorate --relative-date --graph`

`1`	`* 8ca4176 (HEAD -> master) initial commit`

Trivial conflicts

Now lets create a condition for a conflict that is trivial to merge.

Modify file1 in a branch

We first create a branch and add a line:

git checkout -b feature/adding-a-line
echo "line 4 from feature branch" >> file1
git add file1
git commit -m "adding some changes at the end of the file"

1
2

[feature/adding-a-line b24353e] adding some changes at the end of the file
 1 file changed, 1 insertion(+)

And we can see what is in file1:

`1`	`cat file1`

original line 1
original line 2
original line 3
line 4 from feature branch

Modify another line on master branch

Now lets go back to master and add a change to the first line:

git checkout master
sed -i -e 's/original line 1/new change on line 1/' file1
git add file1
git commit -m "changes to line 1"

1
2
3

M	file1
[master 177c8f3] changes to line 1
 1 file changed, 1 insertion(+), 1 deletion(-)

And we can see what is in file1:

`1`	`cat file1`

1
2
3

new change on line 1
original line 2
original line 3

As you can see, we have 2 commits with different changes to file1.

Merging

We only have a conflict if we want to integrate changes from one branch to another.

Lets do this by merging changes from our feature branch to master:

1
2

git checkout master
git merge feature/adding-a-line

Auto-merging file1
Merge made by the 'recursive' strategy.
 file1 | 1 +
 1 file changed, 1 insertion(+)

And we can see what is in file1:

`1`	`cat file1`

new change on line 1
original line 2
original line 3
line 4 from feature branch

As you can see trivial conflicts are automatically merged.

The history now looks like this:

`1`	`git log --oneline --decorate --relative-date --graph`

*   8061307 (HEAD -> master) Merge branch 'feature/adding-a-line'
|\
| * b24353e (feature/adding-a-line) adding some changes at the end of the file
* | 177c8f3 changes to line 1
|/
* 8ca4176 initial commit

What you can read from this history:

HEAD points to the latest commit of our master branch, a merge commit
the commit “changes to line 1 was done on master”
the commit “adding some changes at the end of file” was done on our feature branch
the common ancestor of both commits is the “initial commit”

Non-trivial conflicts

Now lets create a conflicts that you will have to merge manually.

Modify file1 on a branch

We will make changes to line 3 from both our feature branch and our master branch:

git checkout -b feature/change-to-line3
sed -i -e 's/original line 3/line 3 from branch/' file1
git add file1
git commit -m "changing line 3 on our branch"

1
2

[feature/change-to-line3 2e16635] changing line 3 on our branch
 1 file changed, 1 insertion(+), 1 deletion(-)

The history now looks like this:

`1`	`git log --oneline --decorate --relative-date --graph`

* 2e16635 (HEAD -> feature/change-to-line3) changing line 3 on our branch
*   8061307 (master) Merge branch 'feature/adding-a-line'
|\
| * b24353e (feature/adding-a-line) adding some changes at the end of the file
* | 177c8f3 changes to line 1
|/
* 8ca4176 initial commit

The commit at the top is on our new feature branch.

Modify the same line on master

Now lets change line 3 on master:

git checkout master
sed -i -e 's/original line 3/another change to line 3/' file1
git add file1
git commit -m "changing line 3 again"

1
2

[master fc9a98c] changing line 3 again
 1 file changed, 1 insertion(+), 1 deletion(-)

The history now looks like this:

`1`	`git log --oneline --decorate --relative-date --graph`

* fc9a98c (HEAD -> master) changing line 3 again
*   8061307 Merge branch 'feature/adding-a-line'
|\
| * b24353e (feature/adding-a-line) adding some changes at the end of the file
* | 177c8f3 changes to line 1
|/
* 8ca4176 initial commit

Merging our feature branch into master

What happens if we merge our feature branch? We have a conflict that cannot be resolved automatically:

`1`	`git merge feature/change-to-line3`

1
2
3

Auto-merging file1
CONFLICT (content): Merge conflict in file1
Automatic merge failed; fix conflicts and then commit the result.

And we can see what is in file1:

`1`	`cat file1`

new change on line 1
original line 2
<<<<<<< HEAD
another change to line 3
=======
line 3 from branch
>>>>>>> feature/change-to-line3
line 4 from feature branch

In the file1, you can see changes from both branches.

You could at this point, open your text editor and manually resolve the conflict by keeping the text we want, and removing the conflict markers (««<, »»>, ===).

This is what many people do on YouTube when applying patches and trying to resolve conflicts. What you may not realize, is we have tools to do this, like meld, vimdiff, kdiff3.

With our previous configuration, all we need is to run the command:

`1`	`git mergetool`

This screen should appear:

What can you get from this screen:

The middle section is our file1 populated with the common ancestor’s value (original line 3)
The left section (called local) contains our change on our current checkout branch
The right section (called remote) contains the change we made on the branch to merge (our feature branch)
You bring the changes you want using the arrow icons.
You can take changes from both local and remote (you can take more than once )
You can edit the middle buffer

I will take both changes, save and close meld.

After all of this is done, you can commit the changes:

`1`	`git commit`

You’ll be prompted to keep or change the default commit message.

I’m going to accept the default message.

Voila! Now you know how to resolve conflicts using tools.

What if you’re lost

You can use the merge –abort option to abort an ongoing merge and then start again.

Various merge tools

kdiff3

Similar tool than meld. GUI based.

You just need to configure your merge.tool git configuration to kdiff3.

Here is a screenshot of the same merge conflict above, but this time with kdiff3:

Kdiff3 adds a 4th pane, the editing pane.

The base is the common ancestor, the local and remote are like for meld, the changes to the current branch and the branch we want to merge.

Clicking on A/B/C allows you to select which change(s) you want to get. The bottom pane shows you these and also allows you to edit.

vimdiff

Vimdiff is similar to kdiff3 with the 4 panes, except it’s console base. You will find your way if your familiar with vim.

Just set the merge.tool git configuration to vimdiff.

Here is the merge conflict as seen with vimdiff:

Bindings:

move between windows: C-w C-w
navigate to previous / next conflict: [c and ]c
find window numbers: :ls
put the content at point to the destination: dp
get the content from a buffer: dg

Window numbers:

1: local
2: base
3: remote
4: file to edit

Again, the file window can be edited. This is vim, so you can use any vim commands.

Trivial with non-trivial conflicts

In case where you have trivial and non-trivial conflicts, tools react differently:

meld: does not detect trivial from non-trivial
kdiff3: apply trivial changes automatically. Good most of the time, sometime breaks.
vimdiff: does not detect trivial from non-trivial

Thoughts and what’s next

The problem of resolving conflicts may not appear to be that big for an example like above, but over many files and many changes, it’s the best way to resolve them with consistency and quality.

What I’ve learned during this post? I still prefer kdiff3 over meld as a merge tool due to automatic trivial conflict resolution of kdiff3. I haven’t tried Emacs ediff too, will come when I try replacing IntelliJ with emacs lsp-mode in the future.

In my next post, I’ll demo how I organize, apply and merge patches for st using: branches, git apply, and a merge tool.

I hope you learned something :)

This is day 9 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Git patching and resolving conflict (part 1)

Sun, 21 Jun 2020 00:00:00 +0000

I’m creating this post after seeing numerous people suffering and manually addressing merge conflicts while integrating patches with dwm, st, or dmenu code base.

The most common issues seen:

applying patches on latest from master
using patch instead of git to patch
resolving conflict manually

Applying patches and resolving conflict is the land of version control system like git and we have many techniques and tools to make it easy.

This is not specific to suckless.org tools, but I’ll use some example patches to show how to use tools to automate as much as we can.

Note: I’m going to use git, but these techniques are not unique to git.

Definitions

These terms may be oversimplified, but it’s done to keep things simple.

A commit

A registered set of files that changed.

In git, a commit has a unique identifier (a hash code). Git only track changes made to files, so you can’t register a folder in a commit.

A commit has a parent commit (except for the initial commit) and cumulative changes are actually a series of commits, each commits pointing to it’s parent.

branch

A way to parallelized software development. You make changes (commits) to a branch, and you create other branches to separate the work.

In git, the primary development branch is by default called master.

Git branches are actually pointers to the latest commit of the branch. Creating new commits on the branch moves the pointer to the new commit.

Now imagine we want to do some development work while keeping our primary branch stable, we would create another branch:

Now development continues and we have diverging changes but it’s not a problem, we are on different branches.

Fully decentralized

Something many people don’t realize is that git model is decentralized by design. We can have multiple equivalent copies of the same repository. It happens that most people use centralized systems to share code, but this is not mandatory. The whole central system could go and you can still share code and haven’t lost code.

Merge

Now that we are done with our feature, we want to integrate it into master.

We use git merge to merge our myfeature branch to master:

Conflicts

In the merge scenario, all is good if the diverging changes did not touch the same file. If we touched the same file, we end up in conflict. Git and merge tools would resolve trivial conflicts automatically (when we did not change the same lines).

We do need to manually merge conflicts when changes occurs on the same lines.

Manual conflict resolution with 3 panes diff tools

Now, this is manual in the sense that we need to make a decision, but we do not need to copy paste lines, edit the files manually. We have tools for this folks!

There are many tools that we can use to merge conflicts:

vimdiff
emacs ediff
kdiff3
meld
Your IDE might have one
many more

a 3 panes diff tool usually look like this:

|------+-------+--------|
| base | local | remote |
|------+-------+--------|
|    editing buffer     |
|------+-------+--------|

a 3 panes diff tool usually look like this:

1
2
3

|------+---------------------+--------|
| base | local / edit buffer | remote |
|------+---------------------+--------|

Patches

A patch is a diff file (added lines/files, removed lines/files) taken at a point in time relative to a specific version.

One way to share changes is by sending patches. git even has an email client to send patches.

Now if i’m on the latest of myfeature, I can create a patch against master by typing the command:

`1`	`git format-patch master --stdout > myfeature.patch`

Now this patch what taken from the latest commit of master, so it is good practice to add the commit id to the patch. This way people know how to apply the patch.

Applying patches

Now lets say, time has passed, and master has additional changes.

The common mistakes is trying to apply the patch on the latest master branch.

This may work, but as time pass, you may end up with issues.

The right way to apply this patch is to create a new branch from the commit this patch was taken from, and then merge this branch to master.

Thoughts and what’s next

Time to send this. I realize this problem needs to be split in couple of manageable pieces.

This post was to set ground on git terminology, and patching techniques and address one of the problems often seen when patching: patching from the right place in the history.

I see two possible follow-up posts:

merge conflict resolution
real life example using dwm, st, or dmenu

This is day 8 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Dotfiles management using chezmoi - How I Use Linux Desktop at Work Part5

Mon, 15 Jun 2020 10:30:17 -0400

Part 5 of the series How I use Linux desktop at work where I describe how I’m building a new Vagrant configuration to use for coding on Windows Host systems.

This post describes how to automate software installation and manage your dotfiles using chezmoi.

In this post, I’ll also replace openbox with my favorite DM, dwm.

part1 - Basic setup
part2 - X, and WM
part3 - VirtualBox guest additions
part4 - LightDM and autologin

In the post, I’ll cover:

Dotfiles, my dotfiles management history and why I now use chezmoi
Installing chezmoi
chezmoi commands
Install and configure core apps like: dwm, dmenu, st, sxhkd
Integrate with my vagrant setup
Thoughts and what’s next

Dotfiles, my dotfiles management history and why I now use chezmoi

What are dotfiles?

Lets start with some definitions and descriptions

Dotfiles are files which contains application configuration, settings or preferences. Some dotfiles are edited with a text editor, while some are modified directly in the applications that they serve.

Dotfiles usually starts with a period (.) or are with a folder starting with a period, this is where they take their name.

Dotfiles location conventions

The long time tradition is that dotfiles are stored directly in your home folder (~/). This was quite ok in the old days when you were running couple of apps, but nowadays, this is totally unrealistic. In my system only, I count several thousands of settings files. Most applications nowadays stores their dotfiles under ~/.config.

Appears that the ~/.config is part of the freedesktop.org XDG Base Directory Specification under the $XDG_CONFIG_HOME variable. By default, the specs specifies ~/.config. This is rarely changed (never seen it different).

In your own system, you’ll find core utilities dotfiles sitting directly under your home folder, while most applications dotfiles will be under a subfolder of ~/.config.

My dotfiles management history

If you are like me and customize/configure your apps and tools, you will end up with many useful customization you want to carry around with you.

This is where dotfiles management is useful (there is an extensive github page around dotfiles, management tools, and practices).

Personally, I’ve been managing my dotfiles using various techniques and tools:

started by just zipping them and emailing them around. it helped me for a while
Then put them in a git repo. and manually creating symlinks
Moved to yadm management tool with a bare repo, but did not stick with it. Bare git repos could be risky if you are not cautious. Also could not find out how to manage sensitive configuration files
Back to basic git repo with stow using symlinks. This worked well, but still hard to manage sensitive files.
Replaced stow with makefiles, and envsubst scripts to implement templating. This worked well, but tedious to maintain. Also, if I modified the destination, I would have to remember to integrate the changes also in my git repository.

As you can see, all these solutions are usable, but far from working for me. I was still on a search for a simple dotfiles until i discovered chezmoi.

Some key features / benefits that works for me:

Like my latest workflow, it install files instead of creating symlink.
It supports templates
template values can be conditional on the system I install my dotfiles on
supports injecting values from commands like password managers
Can integrate changes on your target files using merge tools like vimdiff

So now I am porting my dotfiles to chezmoi. The remaining of this post will be on my learning journey.

I will share my dotfiles on my github dotfiles repository.

Installing chezmoi

Head over to the Install | chezmoi.io page for all options.

In my case, I’m installing on arch so:

`1`	`pacman -S chezmoi`

You can see if chezmoi is properly installed:

`1`	`chezmoi --help`

Manage your dotfiles across multiple machines, securely

Usage:
  chezmoi [command]

Available Commands:
  add              Add an existing file, directory, or symlink to the source state
  apply            Update the destination directory to match the target state
  archive          Write a tar archive of the target state to stdout
  cat              Print the target contents of a file or symlink
  cd               Launch a shell in the source directory
  chattr           Change the attributes of a target in the source state
  completion       Generate shell completion code for the specified shell (bash, fish, or zsh)
  data             Print the template data
  diff             Print the diff between the target state and the destination state
  docs             Print documentation
  doctor           Check your system for potential problems
  dump             Write a dump of the target state to stdout
  edit             Edit the source state of a target
  edit-config      Edit the configuration file
  execute-template Write the result of executing the given template(s) to stdout
  forget           Remove a target from the source state
  git              Run git in the source directory
  help             Print help about a command
  hg               Run mercurial in the source directory
  import           Import a tar archive into the source state
  init             Setup the source directory and update the destination directory to match the target state
  managed          List the managed files in the destination directory
  merge            Perform a three-way merge between the destination state, the source state, and the target state
  purge            Purge all of chezmoi's configuration and data
  remove           Remove a target from the source state and the destination directory
  secret           Interact with a secret manager
  source           Run the source version control system command in the source directory
  source-path      Print the path of a target in the source state
  unmanaged        List the unmanaged files in the destination directory
  update           Pull changes from the source VCS and apply any changes
  verify           Exit with success if the destination state matches the target state, fail otherwise

Flags:
      --color string         colorize diffs (default "auto")
  -c, --config string        config file (default "/home/benoit/.config/chezmoi/chezmoi.toml")
      --debug                write debug logs
  -D, --destination string   destination directory (default "/home/benoit")
  -n, --dry-run              dry run
      --follow               follow symlinks
  -h, --help                 help for chezmoi
      --remove               remove targets
  -S, --source string        source directory (default "/home/benoit/.local/share/chezmoi")
  -v, --verbose              verbose
      --version              version for chezmoi

Use "chezmoi [command] --help" for more information about a command.

chezmoi use cases

Getting help

Chezmoi binary has a built in help. You can get help for any chezmoi commands using the help command followed by the command you want to get help on.

Example:

`1`	`chezmoi help help`

Description:
  Print the help associated with *command*.

Usage:
  chezmoi help [command] [flags]

Flags:
  -h, --help   help for help

Global Flags:
      --color string         colorize diffs (default "auto")
  -c, --config string        config file (default "/home/benoit/.config/chezmoi/chezmoi.toml")
      --debug                write debug logs
  -D, --destination string   destination directory (default "/home/benoit")
  -n, --dry-run              dry run
      --follow               follow symlinks
      --remove               remove targets
  -S, --source string        source directory (default "/home/benoit/.local/share/chezmoi")
  -v, --verbose              verbose

Get started

If you look at the quickstart, you’ll find that all you need to start is:

`1`	`chezmoi init`

then you can add managed files using the add command.

Now lets take ~/.bashrc

`1`	`chezmoi add ~/.bashrc`

This will create a copy if this file under the chezmoi storage folder ~.local/share/chezmoi/

You can also use the -r option to add files recursively:

`1`	`chezmoi add -r ~/.config/newsboat`

storage location

You can cd to the chezmoi storage folder

`1`	`chezmoi cd`

Here you can do any file operations you want.

Note that file names contains encoding for permissions and have a specific meaning for chezmoi.

Managed files

From the moment you add a file to chezmoi, it gets copied to the chezmoi local directory and it becomes “managed”.

You can find the list of managed files with the command:

`1`	`chezmoi managed`

version control

Now that we have a file managed, the next thing you may want to do is to version control your dotfiles.

All you need to do is to go to the chezmoi storage folder and initialize the repository.

For git, it would be:

chezmoi cd
git init
git add .
git commit -m "initial commit with my bash configuration"

Chezmoi also has a git command:

1
2
3

chezmoi git init
chezmoi git add .
chezmoi git commit -m "initial commit with my bash configuration"

Making changes to managed files

Any changes made outside of chezmoi’s folder will be lost by default.

The best way to edit files is to edit files using chezmoi.

There are really two ways:

call the edit command, like chezmoi edit ~.bashrc/ This command use the EDITOR environment variable to launch your favorite text editor
navigate to the chezmoi’s folder and edit them directly there

Apply changes to managed files

The apply command can install any changes made to managed files on their target (ie: where they are used).

For example, if we changed the ~/.bashrc managed file using the chezmoi edit ~.bashrc/ command, we can install the new version using the command:

`1`	`chezmoi -v apply`

Note: changes made to the targets are overwritten by the apply command.

If you are unclear what changes will be applied, ensure you use the –dry-run or -n option:

`1`	`chezmoi -v -n apply`

What you can also do is call edit with the -a command to apply after editing. You may also like the -d and -p together with -a to diff and confirm apply.

`1`	`chezmoi edit -a -d -p ~/.bashrc`

Updating from remote changes

Here I usually just cd and run git commands.

You can also use the chezmoi update commands which pull changes from git remote and optionally apply changes.

You have choices here.

Integrate changes made to the target files outside of chezmoi ATTACH

There are cases where the target file (ex: ~/.bashrc) is modified outside of the chezmoi repo. In this case, if you dont integrate changes in, you will loose them on the next apply command.

You have a couple of options to integrate these changes:

do it manually. edit both files and copy paste changes
a better way is to merge your changes using diff tools. chezmoi has the “merge” command to do this

Upon calling chezmoi merge on the specific files edited outside chezmoi, it will start a three-way merge on the file, showing the common ancestor, the value on chezmoi and the local changes. The default tool is vimdiff, but you can define your diff3 tool of choice like meld, or kdiff3.

Here is an example of such diff tool.

On the left, my local changes on ~/.bashrc. On the center, the changes within chezmoi.

If I want to integrate my local changes to chezmoi, I just need to click the arrow to bring changes from the left pane to the middle one.

Configuration file

The chezmoi configuration file is optional and located in the ~/.config/chezmoi/chezmoi.toml

You can specify the merge tool used by the merge command like here for meld:

1
2

[merge]
  command = "meld"

Templates

Early on, I mentionned one reason I selected chezmoi is due to it’s templating support.

The easiest way to create a template is by using the auto-templating.

Lets say my ~/.bashrc file is like this:

`1`	`export EMAIL="myemail@athome.com"`

and we want to have different values depending on the system.

So before adding our file to chezmoi, we’ll just configure our local email in the chezmoi configuration file:

1
2

[data]
  email="myemail@athome.com"

And we add our bashrc file with the autotemplate feature:

1
2
3

# -T for template
# -a for auto
chezmoi add -T -a ~/.bashrc

the resulting template stored in chezmoi would be:

`1`	`export EMAIL="{{ .email }}"`

The {{ .something }} gives you access to values within your configuration file that will be replaced during the apply command execution.

You can see a list of pre-defined variables here.

Secrets

Sources for templates can be the configuration file or secret storage tools like bitwarden, gpg, keypassXC, pass, and many more.

See the page keep data private for more details.

Scripts

If you create scripts starting with run_ or run_once_, chezmoi will run them at every apply or once unless changed.

For install software, it’s better to create run_once_ scripts.

What else

The items above are the ones I plan to use the most. But chezmoi has much more to offer and the documentation on it’s website is quite good.

Install and configure core apps like: dwm, dmenu, st, sxhkd

I am using run_once_core.sh to install dwm, dmenu, st, sxdkd, dunst, status bar, and similar tools.

If you want details, just head over to my dotfiles repository.

Integrate with my vagrant setup

To get this integrated with my vagrant setup, I need to add chezmoi to the vagrant provision script, and initialize chezmoi from my git repository.

Thoughts and what’s next

This is the last post of the VM development environment using vagrant (at least for now). The main effort left for me is installing / configuring additional software, also moving some of the vagrant provision script into the chezmoi run_once script.

Some topics for next posts:

applying patches the git way - or how to patch dwm tools the coder’s way
leveraging git history to find hot spot in your source code
test coverage, only half of the answer to quality measure of our automated tests
a series around replacing IntelliJ with Emacs LSP for Java development

This is day 7 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Display Manager With Autologin - How I Use Linux Desktop at Work - Part4

Fri, 12 Jun 2020 21:19:25 -0400

Part 4 of the series How I use Linux desktop at work where I describe how I’m building a new Vagrant configuration to use for coding on Windows Host systems.

This post describes the learning process to setup LightDM with autologin.

As with previous posts, I will go over the research and manual implementation first, then go over how to automate it.

In part 3, I described how I automated the build of the VirtualBox guest additions using vagrant. Part 4 will improve our current box by swapping the display manager to LightDM and setup autologin.

part1 - Basic setup
part2 - X, and WM
part3 - VirtualBox guest additions

In the post, I’ll cover:

Rationale
Installing lightDM, a greeter
Configure autologin
Automating the whole thing
Thoughts and next steps

Steps 1-4 are describing the manual process while step 5 describe how I automated this in the Vagrantfile.

Note: You can find the sources created during this post in my github devbox-arch repository.

Rationale

I want to setup autologin since I want to get in the guest linux box ASAP without unecessary login.

I only want to do this on a vagrant box under certain conditions:

This vagrant box has a virtual NAT network enabled and no port forwarded except ssh using ssh keys setup by vagrant
This box is secured by my host OS credentials. I cannot get in this box without unlocking my Host desktop.

Installing lightDM, a greeter, and XSession

I really like the quality, somewhat simplicity and features that lightDM gives.

The setup I usually put in place with lightDM:

lightDM itself
the gtk greeter. There are better greeter, but this one is simple, available everywhere
a xsession that starts my ~/.xinitrc

In arch, this means installing lightdm, lightdm-gtk-greeter, and xinit-xsession (AUR)

For the first two, its simple:

`1`	`pacman -S lightdm lightdm-gtk-greeter`

xinit-xsession is part of AUR, so it’s a bit more involved as you have to build it from source.

# some required tools
pacman -S git fakeroot

# now clone the AUR git repository
git clone https://aur.archlinux.org/xinit-xsession.git
cd xinit-xsession
makepkg -sic --noconfirm

Now time to enable lightdm:

1
2

systemctl enable lightdm
systemctl start lightdm

You’ll now see the lightdm greeter with the vagrant user pre-selected.

If you want to login, the password is vagrant.

Note that you can select xinitrc as a session in the session selection at the top right of the screen.

Configure autologin

Autoconfig is configured in lightDM main configuration /etc/lightdm/lightdm.conf.

You must set the following two properties to get autologin with xinit session enabled:

[seat:*]
autologin-user=vagrant
autologin-session=xinitrc

Note: you have to uncomment the two properties under the existing seat configuration.

You can restart lightdm to see it in action:

`1`	`systemctl restart lightdm`

You also need to add the autologin group and add user vagrant to this group:

1
2

groupadd -r autologin
usermod -a -G autologin vagrant

Automating the whole thing

All changes were made to the 2-core.sh script.

The interesting part is how to install xinit-xsession and then setup lightdm for autologin.

Installing xinit-xsession using AUR comes to one command:

cd /tmp
    && sudo -u vagrant git clone https://aur.archlinux.org/xinit-xsession.git && \
    cd xinit-xsession && \
    sudo -u vagrant git checkout 7cae213844b && \
    sudo -u vagrant makepkg -sic --noconfirm

Note: makepkg needs to run as a regular user, so you need to clone, and call makepkg using vagrant.

Enabling autologin can be done using sed:

1
2

sed -i -e 's/# *autologin-user=.*/autologin-user=vagrant/' /etc/lightdm/lightdm.conf
sed -i -e 's/# *autologin-session=.*/autologin-session=xinitrc/' /etc/lightdm/lightdm.conf

The sed’s -i option enables in-place editing which change the file specified.

The last part, creating a .xinitrc that calls openbox:

cat <<EOT > ~vagrant/.xinitrc
#!/bin/bash
exec /usr/bin/openbox
EOT

chown vagrant:vagrant ~vagrant/.xinitrc
chmod +x ~vagrant/.xinitrc

Here, the file is created as root, and then ownership is transferred to the vagrant user.

Thoughts and next steps

I hope I am able to demonstrate how easy it is to provision a Linux development VM using vagrant and some unix scripts.

The hardest part is done, getting a foundation to install development tools and configuration.

In next post, I’ll configure this VM using chezmoi. This will allow me to bring my dotfiles and installation scripts.

When I started this blog, I was hoping it would be useful to others. Now I see that it may be as useful to me. It makes me thinks of my goals, think about improving, it makes me learn.

I hope this series inspires people to shape tools to meet their needs.

This is day 6 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

How I use Linux desktop at work - Part3 - Guest additions

Wed, 10 Jun 2020 22:00:24 -0400

Part 3 of the series How I use Linux desktop at work

Just to recap, this series is describing how I leverage vagrant and VirtualBox to provision development VM on my work PCs. I’m also taking the opportunity to build them from scratch with the tools I use at home.

part1 - Basic setup
part2 - X, and WM

In the post, I’ll cover:

reasons why I ended up compiling the guest additions
installing required tools
mounting drive
building guest additions
automating the whole thing

The end result will make the X guest screen resize to the host window size.

Steps 1-4 are describing the manual process while step 5 describe how I automated this in the Vagrantfile.

Note: You can find the sources created during this post in my github devbox-arch repository.

Reasons

During part 2, I spent most of my time trying to get X guest screen to resize automatically. I tried many things like, installing the dkms package, downgrading my virtualBox to the exact version of guest additions available in arch. All of this time without positive results.

Let me know if you know how to solve it without building guest additions.

Installing tools

guest additions needs which, gcc, perl, linux-headers packages in order to be compiled.

you can do so by running these commands.

`1`	`pacman -S which perl gcc linux-headers`

Mounting drive

From the devices menu, select the “insert guest additions CD…” which is the equivalent of inserting the CD in the virtualdrive

Now mount this ISO to the /mnt mount point

`1`	`mount /dev/sr0 /mnt`

Building guest additions

Now you just need to go to the /mnt path and run the installer:

1
2

cd /mnt
./VBoxLinuxAdditions.run

Accept the installation.

Then reboot and voila!

Automating the whole thing

Now, we do not want to do this manually every time, lets automate this.

This comes in two parts:

binding the ISO to the DVD drive
mounting and building

Binding the ISO to the DVD drive

Now automating binding of the ISO is fairly easy with vagrant. You just have to use the modifyvm to create a CD drive and tell to “put” the ISO file in the drive:

`1`	`vb.customize ['storageattach', :id, '--storagectl', 'IDE Controller', '--device', 1, '--port', 1, '--type', 'dvddrive', '--medium', 'C:\Program Files\Oracle\Virtualbox\VBoxGuestAdditions.iso']`

Mounting and building

Like I described in part 2, we will update the “2-core.sh” script to build the guest additions.

1
2

mount /dev/sr0 /mnt && cd /mnt && ./VBoxLinuxAdditions.run -- --force
umount -f /mnt

Rebooting

I use the vagrant-reload plugin to reboot while provisioning.

Once installed, you can reboot the VM with:

`1`	`config.vm.provision :reload`

Value of all this

At this stage, with just a single vagrant up command, I can automatically provision a Linux VM with X configured and a basic window manager.

This will serve as a foundation to my work/home dev boxes.

What is coming next

Some topics for the near future:

Swap the X session manager to an auto-logger
Configure dwm/st/sxhkd/dmenu using chezmoi dotfiles management to provision everything
Coding tools installation

This may just be one post, we’ll see.

I hope this series is of some use to others and inspire people to use tools that suits their needs.

This is day 5 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

How I use Linux desktop at work - Part2 - Window manager and GUI

Tue, 09 Jun 2020 16:31:24 -0400

Part 2 of the series How I use Linux desktop at work

In the post, we go over:

creating a basic arch VM
configure VM
install X related packages
Install one Window Manager (lxde)

The end result is a versioned configuration that can provision a basic arch VM with X and a window manager.

Note: You can find the sources created during this post in my github devbox-arch repository.

Pre-requisite

Stuff assumed to be configured before starting and already covered in part1:

VirtualBox
vagrant

Also I make use of git bash to version control my configuration and also to run vagrant.

Note: Many commands below will run fine on git bash, but not on powershell. Just take this into consideration.

Creating the basic arch VM

At this stage, we’ll create a bare minimum vagrant file that creates VM based on arch.

I use the official arch vagrant box called archlinux/archlinux.

In my git bash, I type:

1
2
3

mkdir devbox-arch
cd devbox-arch
vagrant init archlinux/archlinux

That’s it. Lets boot it up and connect to it:

vagrant up
vagrant ssh
uname -a
exit

setup a git repository

Might as well create a git repository right away. This will create a safe environment to break things and revert if needed.

git init
echo ".vagrant/" >> .gitignore
git add Vagrantfile .gitignore
git commit -m "basic arch console config"

configure VM

The VM we just created is good enough for console tools, but if we are going to use this for a development VM running X, we need to allocate more resources, setup graphic properties, setup host/guest integration.

Here is my vagrant file, it includes variables to define various properties like memory, CPUs, monitors. My objective is to be able to quickly adapt to the resources available on the host computer I use.

Couple of things to note about the vagrant file below:

ensure that you don’t over allocate resources. Don’t assign all your CPUs and ram to the VM. If you launch VirtualBox UI, can you see safe limits in the settings menu.
VM customization below starting with vb. are specific to VirtualBox, if you use a different vagrant provider, these options won’t work.
Clipboard is setup bidirectional between host and guest
The host folder C:\Users\\src (~/src in git bash) is mapped to the guest folder ~/src. This is usually the source folder I use to share sources.

# -*- mode: ruby -*
# vi: set ft=ruby :

NB_MONITORS = "1"
NB_CPUS = 6
MEMORY = "8192"
VRAM = "64"

Vagrant.configure("2") do |config|
  config.vm.box = "archlinux/archlinux"

  config.vm.provider "virtualbox" do |vb|
    # Display the VirtualBox GUI when booting the machine
    vb.gui = true
    vb.name = "devbox-arch"

    # resources
    vb.memory = MEMORY
    vb.customize ["modifyvm", :id, "--cpus", NB_CPUS]

    # graphics
    vb.customize ["modifyvm", :id, "--monitorcount", NB_MONITORS]
    vb.customize ["modifyvm", :id, "--vram", VRAM]
    vb.customize ["modifyvm", :id, "--graphicscontroller", "vmsvga"]
    vb.customize ["modifyvm", :id, "--accelerate3d", "off"]

    # Customise audio
    vb.customize ["modifyvm", :id, "--audioout", "on"]
    vb.customize ["modifyvm", :id, "--audio", "dsound"]
    vb.customize ["modifyvm", :id, "--audiocontroller", "hda"]

    vb.customize ['modifyvm', :id, '--clipboard-mode', 'bidirectional']

  end

  # mount my Users\<>\src folder on host to /home/vagrant/src
  config.vm.synced_folder ENV['USERPROFILE'] + "/src", "/home/vagrant/src"
end

You may want to destroy your VM (vagrant destroy) and recreate it (vagrant up).

Setting up additional VM options

You may want to add / change the configuration described above. The vb.customize configurations above use the VBoxManage.exe command from the C:\Program Files\oracle\virtualbox folder.

Call VBoxManage to get more details or you can head over to the VBoxManage documentation.

configure arch

My strategy to provision the configure the OS is to keep the configuration out of the VagrantFile and create a series of shell scripts called by the VagrantFile.

This is done by the provisioner options.

Here is an example calling a specific script as root (by default).

`1`	`config.vm.provision "shell", path: "1-setup-arch.sh"`

Followed by a vagrant command to run the provision script:

`1`	`vagrant provision`

The config.vm.provision instruction supports inline scripting as well as normal user provisioning.

The setup arch contains instructions to:

setup timezone and locale
configure pacman to use a regional mirror
update the package list and update with any new package versions

Getting X environment working

Same as with arch config, I’m using a script to setup X and related software.

`1`	`config.vm.provision "shell", path: "2-core.sh"`

Again followed by:

`1`	`vagrant provision`

Some objectives for the core provisioning script is:

install X, it’s drivers, a window manager
setup audio
install virtualbox guest additions with X support

Now you should see a GUI login prompt with the vagrant user selected. You can enter the super secret password (ie: vagrant).

Warning: The vagrant account is safe as long as you don’t weaken the existing setup (like: enable services using password authentication).

Time to speed up

I’m running out of time, and as I previously posted, I prefer something working to something perfect.

Here are the things I’ll be improving and describing in future posts:

setup an auto logger to get in the vagrant account instead of a session manager
add additional core apps like vim.
replace lxde with WM of choice (dwm) with related tools/configuration

So in a nutshell, the 2-core.sh script will evolve.

Things that could go wrong

Got two issues while building this configuration:

pacman update fails due to bad mirror.

This is why I’ve been selecting the first mirror that works for me
guest OS resolution does not adjust with the VM window size. I wasted quite some time trying to find why.

This issue is due to incompatibility with your VirtualBox and the install guest additions. you have two choices here:
1. install the exact same version as the virtualbox-guest-utils package. Does not always work.
2. build the guest additions from the VM iso by selecting “insert guest addition” in the VirtualBox menus. I will create a new post how I’ve done this.

where to find sources

This is becoming my default setup for both home and work.

You will find this particular configuration in my git repository: devbox-arch.

If you look at the master branch, you’ll most definitively find something different as this configuration will evolve over time.

What is coming next

Some topics for the near future:

New post to cover how to get window resizing by installing/compiling the guest additions
Using chezmoi to manage dotfiles and tools installation

I hope this series is of some use to others and inspire people to use tools that suits their needs.

This is day 4 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

How I Use Linux Desktop at Work - Part 1 - Basic setup

Fri, 29 May 2020 11:31:24 -0400

Most coders is large companies are facing the same situation: We develop for Linux servers, but we are running on Windows or MacOS boxes.

Things are working mostly the same at best, at worst, it’s broken.

We need a solution and Vagrant is a quick way to provision a Linux VM on your local desktop.

This will be a series of posts starting with these 4 topics:

Basic setup
Getting basic X setup
Make it your own. Dotfiles and development environments
Variants using boxes from Linux OS

Why Vagrant

For those who don’t know, Vagrant by HashiCorp is a VM provisioning tool targeted at spawning software development boxes. I’ve used Vagrant for the last 3-4 years to quickly get a Linux development VM running on my Windows Desktop.

Vagrant support various VM/Hypervisor platforms like libvirt/KVM (Linux), MS HyperV (Win), Oracle VM VirtualBox (both), VMWare

Vagrant primary file is a VagrantFile which describes how the VM must be configured. It goes from which OS to use, how many memory/cpu to use, disk size, shared drive between Host and Guest, and much more.

There are couple of reasons to use this setup to configure a Linux development environment:

Quick and reproducible
Can be shared with colleagues and friends
Isolate the environment from your host OS. Can become project specific setup.
Develop on the same OS as your target OS
You can use vagrant to setup a set of VMs that works together (things like a Kubernetes cluster)

Virtualization

There are couple of choices of virtualization platform:

Virtualbox
VMWare
Docker
HyperV
WSL (beta)

In most cases, Virtualbox is the best solution due to it’s Guest Additions integration and USB2.0/3.0 support.

In short, why others are lesser capable IMHO:

VMWare: cost something. You can get away by running only the player, but it’s not as capable than the regular product or Virtualbox
Docker: on Windows, docker runs on a HyperViser, I find this adds a layer of complexity to the VM approach. I also use docker daily, and running docker in docker can work, but it needs special configuration
HyperV: The configuration is quite similar to Virtualbox. Capable configuration UI, and quick fast execution. Where it fall short compared to VirtualBox is it uses RDP instead of a seamless integration with the Host we get with Virtualbox. If it does not bother you, then HyperV is a valid option, and maybe a faster one.
WSL: First, support for WSL in Vagrant is beta. It’s also only text based (I know you can go around it). Not really giving what you need compared to the alternatives
WSL2: There are some tickets open to support WSL2. This may be a viable option in the future. Maybe this will bridge the gap between HyperV and Virtualbox with regards to seamless integration with the host OS. Too early to tell.

Basic setup

The aim of this first part is to get a prompt on the Guest OS of choice, in this case Arch, but it could be any other Linuxes.

Enable virtualization

To make things fast, we need to use Virtualization implemented using Hardware virtualization capabilities of your PC. In theory, software based virtualization could work, but it’s far from being usable if you want to run X and a window manager (ie: way too slow).

Most PC these days support Hardware virtualization (even my old x220 from 2012), but it may not be enabled.

You can enable virtualization by going to your BIOS and enabling hardware virtualization. It can have various names, VT-x, Virtualization, or something similar. If not enabled, Virtualbox wont be able to give you hardware virtualization. If this is on your work PC, you may need your IT department to enable this if they password protected your BIOS.

Install Virtualbox

Install Oracle VM VirtualBox. In my case, I install the VirtualBox for Windows Host.

You can optionally install the Oracle VM VirtualBox Extension Pack which gives you host bridging with USB 2.0/3.0 devices. This is required if you want to pass-through a USB 2.0 or 3.0 device connected to your host directly to your VM.

Vagrant

Vagrant requires newer version of Powershell to be able to install. Windows 10 has the right version and I’ll assume you are on Windows 10 since other versions are EOL.

Head over Vagrant by HashiCorp and download vagrant.

Installation should be trivial.

Vagrant Box selection

Time for box selection. A vagrant box is a pre-configured bare VM that you can download and start. It’s the VM equivalent to a docker image.

Head over to Discover Vagrant Boxes - Vagrant Cloud page to search for boxes.

Like any community contributed software, take special care to select a box you can trust.

Disclaimer: The box I’m going to select here is based on my own evaluation. I’m not flawless, and also things could change over time (ie: what is safe today may not be in the future). You should do your own assessment. You’ve been warned, don’t blame me if this breaks your system.

I would recommend to take boxes from reputable sources, or build you own (which is way beyond the scope of this series).

To test the setup, I’ll just select a small VM for alpine: generic/alpine38

VagrantFile

Vagrant is configured in a similar way to docker, with a VagrantFile used to “build” your VM.

To create the simplest VagrantFile for a new setup, open a cmd prompt or powershell and type:

1
2
3

mkdir test-vm
cd test-vm
vagrant init generic/alpine38

BTW, Vagrant is written in ruby and the VagrantFile is a ruby file.

Here is the equivalent of your Vagrantfile:

1
2
3

Vagrant.configure("2") do |config|
  config.vm.box = "generic/alpine38"
end

Booting the box

Now that your configuration file is created, run the following command in a powershell in the created directory:

`1`	`vagrant up`

In the end of the process, you should have a VM provisioned and started.

Connect to your box

From the same vagrant folder, you can connect to your box using ssh with keys generated for your:

`1`	`vagrant ssh`

You are now connected to your box using SSH.

You can now disconnect from SSH (ie: exit or ^d)

You may have noted that vagrant mounted /vagrant to your local vagrant folder. In your box, you can share files from that folder. There are ways to mount additional folders and we’ll go over this in a later post.

Shutting down

Another useful command, is you can tell vagrant to shutdown the VM using:

`1`	`vagrant halt`

Destroying your box

Since this is just a test box, you may want to delete the VM:

`1`	`vagrant destroy`

What I get out of this

As you can see, Vagrant helps you setup a minimal Linux OS running in a VM in just a few steps.

This is way more efficient than installing from ISO and going through the installation.

Coming next will be part 2, addressing basic X setup as well as tweaking the VM configuration using the same VagrantFile.

This is day 3 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

Blogging Setup

Sun, 24 May 2020 21:25:51 -0400

In this second post, I’ll go over my thought process to create / publish a blog using markup languages and couple of tools.

The objective of my setup was to be done quickly so I can write something else than software configuration :)

My hopes for my blogging setup:

something I can use from day one, get out of the way, and allow me to improve at a later time
automated build and deploy
use a markup language like commonmark/markdown, org-mode, or Asciidoc. Preferably org-mode since I want to use org babel with code blocs.
simple
1. presentation in a super simple CSS
2. use HTML5
learn something I haven’t done before

Spoiler alert: it’s possible to get up and running in one day, if you accept some compromise and don’t get stuck on trying to make everything perfect.

Disclaimer: There are much easier way to get started using platforms like Wordpress or write.as. This solution is tailored to what I’m comfortable doing, it may not be the right solution for you.

Disclaimer2: This is not a howto guide, but hopefully source of inspiration.

Build the site

Couple of options I’ve looked at:

use something like pandoc with couple of shell scripts or Makefiles Great to convert single files. Could have worked for a 1 day setup, but not necessarily easy to get going on a free CI platform. Also a I bit too custom to my liking if I want to create templates, archetypes. All possible, but I suspect too needy.
use org publish with plain org files Not too convinced with this one. It feels like pandoc, but in Emacs :). Easy to start, but would not be great to integrate with a CI platform.
use the great export plugin ox-hugo - Org to Hugo exporter from Kaushal Modi Would work better for me since this is a bridge between Emacs/org-mode and a static site generator. The main downside to me is that I did not used Hugo before, and this creates abstraction on top of Hugo. Would Be a good addition once I know Hugo.
use a static site generator like Jekyll or Hugo Since I have some hope to use ox-hugo in the future, I did try to get Hugo working for my blog. Hugo also support org-mode in addition to markdown.

Prototyping

Followed the Hugo getting started and got a two dummy post created.

This is really easy to get started. Here are the main steps:

create a site
associate a theme
create some posts
run Hugo in server mode and open up your browser to http://localhost:1313

That’s all you need to get started.

The hardest part is yet to come. Teeming and Rendering.

I really like clean and simple. Now there are many nice clean/simple theme on Hugo site, but the ones I liked had big CSS and many even made use of JavaScript. Also, none made use of simpler elements defined in HTML5.

What I was looking for was in the spirit the opinionated thebestmotherfuckingwebsite.

Found this now unmaintained Hugo theme: https://github.com/mpcsh/hugo-theme-motherfuckingwebsite

Made my own version to be able to make it work:

change the navigation menu: add a link to the post RSS feed, merge about and contact together
issues with generating the index list. looks like it is sensitive to folder name
index template is using description front matter data instead of summary

The templates and CSS are quite simple but not HTML5.

At this point, it’s good enough visually, on my laptop… No time to waste, next step!

Deployment

Some hope for deployment:

integrates with either GitHub or Bitbucket (hosted accounts I have)
build my Hugo site from sources on some my public repositories.
support custom domain
something I would not have to read much about

Couple of options comes to mind:

github pages
gitlab pages
netlify

Here, my objective is to get it done ASAP. so no A/B comparison of which on is best. The first hit that seems to work is king :).

Search string from the first thing that comes to my mind: hugo publish netlify github First hit: Host on Netlify | Hugo

That’s it, I have my solution :)

Followed the guide, starting with the dummy site above. I ended up swapping it with this a new github repository benoitj/blog.benoitj.ca: my blog sources.

Now each time I push new changes, I can see it on the generated netlify domain.

domain name

Now, lets get a custom domain setup.

I already own one, so after a bit of research found the Netlify guide to Configure external DNS for a custom domain | Netlify Docs

Finding the doc was the hardest part, Netlify appears to sell domains in addition to website hosting. So the first few guides are over setting up Netlify DNS.

If you are in the same situation, you will have to setup a CNAME pointing to the Netlify domain name assigned to you. Don’t forget this will take time for the DNS entry to propagate over the Internet and work (from minutes to hours).

Improvements since the first post

Broken links

Someone spotted a broken link in my about page. Turns out the default version of Hugo used on Netlify does not properly support org-mode.

I updated with success the ENVIRONMENT variable HUGO_VERSION to 0.71.1 and it did fix the issue.

I ended up creating a netlify.toml in my site root to control the Netlify build.

Here is the minimum setup in netlify.toml:

[build]
publish = "public"
command = "hugo --gc --minify"

[context.production.environment]
HUGO_VERSION = "0.71.1"
HUGO_ENV = "production"
HUGO_ENABLEGITINFO = "true"

Enable broken link detection on netlify

It’s now possible to enable checklinks plugin on netlify.

Since I already have a netlify.toml config to control the build, it’s just a matter of adding this to it:

1
2

[[plugins]]
package = "netlify-plugin-checklinks"

Site improvements TODO

Here is my unsorted list of things I may improve:

~~Broken link detection. Try to enable the link check plugin (beta) on netlify.~~
Create taxonomy pages or list. I’m tagging my posts, but nothing visible yet.
improve how code is displayed. probably need to play with the template / CSS
Will try HTML5 templates and make changes to the presentation in the CSS. I’ve got a theme to get me started, but not sure I’ll keep it. Possibly try new.CSS
enable some sort of spellcheck approval on my posts.
not satisfied with the auto summary formatting. I ended up marking the summary to get formatting right.
relationship between summary, RSS and index page. Maybe RSS should have Content instead of summary but keep index with shorter summary.
see if ox-Hugo helps once I really understand how Hugo works.

Some reflection

I did managed to get out of the software / technology faster than I thought.

The danger for me was to spend all my time customizing tools and not writing a single post. I would have learned something for sure, but missed other learning opportunities.

I hope others can find this helpful.

I’m thinking, if I reach 10 posts in the next 2 months, I’ll be on the right track :)

Coming next

Some topics I may write about

Detecting possible broken SRP using your git history
Automated testing quality. Why coverage is not enough.
how to resolve git conflicts. One possible example: merge conflicting patches on a suckless tool like st or dwm.
note taking setup I use (org-mode, org-roam, deft)
Replacing IntelliJ as my daily driver IDE with Emacs LSP Java
plain text diagrams
- plantuml / graphviz
- C4 models macros for plantuml
- branching models
- network diagrams
homelab
linux at work
possibly some backyard work I’m currently working on.
maybe some electronics projects I have in my queue

This is day 2 of my #100DaysToOffload. You can read more about the challenge here: https://100daystooffload.com.

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!

I should try blogging

Sun, 24 May 2020 00:02:13 -0400

Like many, I’ve been thinking about sharing using a personal blog. Like many, I’ve been procrastinating about it :).

I can thank the movement #100DaysToOffload initiated by Kev Quirk on fosstodon to get me out of my lazyness.

Im hoping to share whatever interest me, but mostly:

coding
linux, apps I use
electronics and games
anything I am learning right now, from food, to building house stuff

I intent to post twice a week. We’ll see how long I can keep the blogging procrastination beast away :)

As a constant learner, constructive feedbacks are always welcomed. See my about page for contact info.

This is Day 1 of my #100DaysToOffload

← craftering →

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!