Attempt</th>	Time (in sec)</th>	Remarks</th>	Commit</th></tr></thead>
1</td>	10</td>	reference</td>	link</a></td></tr>
2</td>	20</td>	test `sleep</code></td>`	link</a></td></tr>
3</td>	30</td>	failed to mount at `/userdata</code></td>`	link</a></td></tr>
4</td>	41</td>	kernel panics</td>	link</a></td></tr>
5</td>	41</td>	abandoning this approach</td>	link</a></td></tr>
6</td>	21</td>	couldn't mount `/dev</code></td>`	link</a></td></tr>
7</td>	35</td>	`/dev</code> is mounted</td>`	link</a></td></tr>
8</td>	60+</td>	Gentoo boots</td>	link</a></td></tr> </tbody></table> At this point, I found out that the kernel could mount the userdata partition. I also realised that I was not getting last_ksmg because the kernel was not panicking and instead just `exit</code>ing. So I tried to get the last_kmsg</code> next.</li>` `I tried to forcefully panic the kernel. And got the last_kmsg</code> and console-ramoops-0</code> in the next boot. But they didnt have the parts where kernel logged the Preinit messages. So I thought that giving them a higher priority would make them get picked by the last_kmsg</code> and it didn't help either.</li>` `Finally I abandoned this approach.</li>` `Then I looked into mounting the pseudo-filesystems. I realised that the kernel could not mount /dev.</li>` `I realised that using tmpfs</code> instead of devtmpfs</code> mounts the /userdata</code>. What I did not know at that time that my kernel did not have the options for devtmpfs</code> enabled in its defconfig</code>, this all could be avoided.</li>` `/dev</code> mounts now, but we have to populate it as well.</li> </ol>`I did not have just 8 attempts, the commit history</a> would show that I forgot that my kernel was SELinux Enforcing, I had to change that among many other things. Robert Landley's blogs</a> on initramfs helped me get to know the very basics of an initramfs.</p> Final comments and What to follow next</h2> My sar-preinit</a> repository holds the current workflow of getting a SAR-enabled device boot Gentoo in the Android based device. The Makefile is pretty self-explanatory but still, if you want to follow the process:</p> Install Gentoo in you phone following this blog</a>.</li> Extract the boot.img</code> from you Android phone. At the time of writing this blog, we dont have changing selinux</code> modes automated. Please make sure that you do that first by adding androidboot.selinux=permissive</code> string at the end of the kernel commandline in bootimg.cfg</code>.</li> Rename boot.img</code> to ogboot.img</code> and place it inside the artifacts folder.</li> Fire up a terminal and run</li> </ol> make</span> bootimg_chute</span></span></code></pre> Flash the boot_mod.img</code> created inside the out</code> directory on your phone, preferably using fastboot boot</code>, if you're not that confident. fastboot boot</code> somehow doesn't work for my device, but that's going off-track.</li> Since we haven't figured out the Android in LXC part yet, the best you could do to check your Gentoo boot, is to reboot into the recovery, fire up adb and check the /data/gnu/var/log/</code> directory for dmesg</code> or any other logs.</li> </ol> I plan to merge this entire workflow into SharkBait, so that you don't have to go through the hassle of doing it all.</p> I should have already mentioned this somewhere, but if I didn't, I should tell the readers that all is based on the SharkBaitOS</a> established mostly by @KireinaHoro. The preinit</code></a> I used as a template was the original one from SharkBaitOS. In order to have a better understanding about this project I would recommend the readers to have a look at KireinaHoro's Blogs</a> as well.</p> In my next blog, I will discuss about getting Android 9+ to work inside a LXC-Container, in the Gentoo land. I have yet to achieve doing that myself. Hopefully we will reach there soon.</p> Until then, Happy Hacking !</p> UART on Xiaomi Redmi Note 7 2020-06-02T00:00:00+00:00 This blog is a short sequel to my previous blog</a> where I discussed why I needed to access a serial console on Redmi Note 7 (I'll refer to it using its codename, lavender here onwards). I should give a disclaimer that this blog is one dead-end in the GSoC2020 series. But it could always help newcomers and other curious souls.</p> The Need of a Serial Console</h2> My first attempt at porting SharkBait for SAR failed. I was confident about the approach but had trouble finding out what went wrong. If only somehow I could get the kernel logs before my phone went into the bootloop, I could quickly provide fixes.</p> Fortunately that is when I found this article</a> from the awesome guys at PostMarketOS. My device had a serial console built in. Accessing it via UART could provide the logs I required.</p> So I followed the process described there and disassembled my phone, soldered the required jumper wires to the GPIO pins mentioned in the article and was ready for connecting it to a convertor.</p> Please excuse the potato quality of the upcoming images.</em></p> Opening the back </figcaption> </figure> </div> Soldered the wires on the GPIO pins </figcaption> </figure> </div></p> PL2303HXA</h2> After that, I connected the jumper wires to the PL2303HXA convertor I bought and quickly fired up a screen</code> session, excited to look at the console output.</p> But there was no output</a>. You can see that the convertor's RX LED is blinking when the phone boots up, but there is no output on the console end.</p> I consulted around with people at PostMarketOS and the SharkBait IRC chat. KireinaHoro asked me to grab a multimeter and check out the voltage the UART operated on. The GPIO pins were at a voltage difference of 1.8V with the ground. I came to the realisation that the PL2303 could not interpret signals with voltage that low. So I decided to buy another convertor, about which I was sure to read signals at 1.8V.</p> FT232R</h2> FT232R based convertor </figcaption> </figure> </div> Final Product </figcaption> </figure> </div></p> With my fingers crossed, I fired up another screen</code> shell, and finally got some</em> output. But all I got were the logs thrown by the Bootloader. I needed the kernel logs because the init</code> is launched by the kernel. And so yet again, I went to chat with Alexey from PostMarketOS who is working on lavender. He informed me that the vendor's downstream kernel for lavender would not give any UART logs for some reason (related)</a>. To get the kernel logs via UART, we would have to use the mainline kernel, but that would mean no android for us.</p> So I had two options :</p> Work on bringing UART to lavender by patching the vendor's downstream kernel. But this would take a lot of time from a very</em> beginner kernel hacker like me. Also it would be rather too off-topic from my original GSoC journey.</li> Abandon UART altogether. I had some very hacky debugs in mind which I could use. It would take some time to yield a fruitful result but this was a lower hanging fruit.</li> </ul> And so, I chose the second option.</p> Which I am very glad to share, worked out. As of yesterday, I was finally able to boot Gentoo via the preinit, but all that's for another blog (coming tomorrow).</p> Android Boot High Jinks: And What it Means for SharkBait 2020-06-01T00:00:00+00:00 Presently, SharkBait aims at devices launching with Android version lower than 9. I will discuss few ways we could the port this setup to newer devices, whose boot mechanisms are different. We shall also address the boot process of SharkBait and what System-As-Root actually is.</p> The Boot process of present-day SharkBait</h2> Swapping the Android's init</code> with preinit</code></a></p> We swapped the init</code> executable present in a functioning boot.img with KireinaHoro's preinit</code>.</li> Preinit was responsible for early-mounting partitions and finally switch_root to Gentoo's init, thus firing OpenRC.</li> The "swapping" was essentially replacing a normal Android boot.img</code>'s ramdisk</code>'s init with preinit</code>, and then flashing the new boot.img</code>.</li> After OpenRC is fired, it starts the lxc.android service.</li> </ul> </li> The Android lxc container.</p> Prior to swapping inits, we established an lxc container in the Gentoo userland.</li> The Android rootfs (at that time it was ramdisk</strong> inside the boot.img</em>), is extracted in the lxc container's rootfs folder.</li> Android boots inside the container easily.</li> The lxc container's config</code> might provide some insights as well:</li> </ul> </li> </ol> lxc.uts.name = android</span></span> lxc.rootfs.path = /var/lib/lxc/android/rootfs</span></span> lxc.init.cmd = /init</span></span> lxc.net.0.type = none</span></span> lxc.autodev = 0</span></span> </span> lxc.hook.pre-start = /var/lib/lxc/android/pre-start.sh</span></span> lxc.hook.post-stop = /var/lib/lxc/android/post-stop.sh</span></span> lxc.cgroup.cpu.rt_runtime_us = 950000</span></span></code></pre> This is just a gist of what happened. There was a lot more involved. Gentoo's fstab</code> was tweaked such that it mounted the device blocks and binded them to the container for Android's use. Also KireinHoro managed to make a working real-time group scheduling model</a> by the use of cgroups.</p> Changes in Android 9+ devices</h2> Well, Google has made it very confusing to follow their language in the AOSP docs</a>, I found that Magisk's Documentation</a> is very insightful. Basically, Android Kernels dont use ramdisks as their rootfs</code> now. The older ramdisks</a> found inside a boot.img</code> are now merged inside system.img.</p> This poses some challenges</h3> What do we use now as lxc-container's rootfs</li> If the kernel does not take a ramdisk inside the boot.img</code> as its rootfs, how do we perform the "init</code> <-> preinit</code> swaps".</li> </ul> Turns out Magisk already has some workarounds.</strong></p> Magisk's approach to System-As-Root</h3> In order to circumvent the new system.img-as-rootfs approach, Magisk hexpatches the device's Kernel to use the ramdisk</code> inside boot.img</code> as rootfs. The hexpatch essentially is just replacing the skip_initramfs</code> with want_initramfs</code> (could be any 4 lettered word instead as well) from the Kernel commandline. But replacing it from the extracted bootimg.cfg</code> of a initrd.img</code> did not work for me. So I stick to using hexpatching using magiskboot</code>.</p> Then Magisk patches the original boot.img</code> by adding a new ramdisk.cpio</code> incpio inside it. The new ramdisk</code> has magiskinit</code> disguised as init</code>. Source</a></p> #</span> original System-As-Root ramdisk #####</span></span> .</span></span> </span> #</span> ramdisk after flashing Magisk #######</span></span> .</span></span> \|</span>-</span> .backup/</span></span> `</span>-</span> init</span></span></code></pre> magiskinit</code> is responsible for :</p> Early mounting required partitions. On system-as-root devices, we will switch root to system.</li> Injecting magisk services into init.rc</code>.</li> Loading sepolicy either from /sepolicy</code>, precompiled sepolicy</code> in vendor, or compiling split sepolicy.</li> Patching sepolicy rules and dump to /sepolicy</code> or /sbin/.se</code> and patching init or libselinux.so</code> to load the patched policies</li> Executing the original init to start the ordinary boot process</li> </ul> This is conceptually somewhat similar to KireinaHoro's approach at Preinit.</p> My approach to revise SharkBait</h2> I have two approaches. Both approaches have some commonality. We apply the same hexpatch which Magisk applies, to the custom kernel which is already lxc-enabled. This forces the kernel to use the ramdisk</code> inside the boot.img</code> as its rootfs</code>. Then we package a new boot.img</code> which had KireinaHoro's preinit</code> as its init</code>. I assume, this enables the kernel to fire up Gentoo's init and eventually start the lxc-container on the way while Gentoo is booting up.</p> We make the edits to the fstab</code> at the /vendor</code> partition (which as you might guess, is another change made by google), in order to provide Android with the correct mounts in Gentoo's userland. Another change made by google was the decision to parse the cgroup mounts using /etc/cgroups.json</code> instead of the original parsing init.rc</code> approach. Source</a>. Fortunately this might not affect us as we dont need to mount any new cgroup. We make edits in the init.rc</code> for bind-mounting the container's pseudofs(s).</p> 1. w/ Magisk</h3> The differences arises in what we use as the rootfs</code> for our lxc-container. The approach I already followed, required using only magiskinit as the rootfs</code>. I expected magiskinit</code> to follow its approach and eventually mount all the required device blocks inside the Android userland. Then finally switch to /system</code> as the rootdir. This was the method which failed, causing my device to bootloop.</p> 2. w/o Magisk (untriaged)</h3> This approach requires to use entire system.img</code>, extracted, as rootfs</code>. The init</code> used here is the Android's original init</code>. Since every file required of the merged ramdisk is present in the rootfs</code>, I expect that init</code> fires up smoothly.</p> Except that since system.img</code> should already be mounted by the kernel in a system-as-root device, We might have to mount it manually. Source</a>.</p> P.S. This article was already written in drafts a few weeks ago, the next paragraphs serve no purpose other than that of history. I will be writing more articles explaining what happened afterwards in order to provide a smoother and realistic reading experience.</em></p> Without the use of a serial console, this whole "trace-fest" is difficult to understand. Finding out what went wrong, is on an entirely different level.</p> We have no dmesg logs as they get overwritten by the recovery boot, which in turn is what I require to check the dmesg logs. Since the kernel does not panic and the init</code> just exit</code>s, we have no last_kmsg</code> or console-ramoops</code> logs either. Fortunately spending a lot of time researching I found this</a>. My device has a serial-console baked in. Although accessing it would require taking out the phone's glass back, it is worth the fruit.</p> Installing Gentoo on Android 2020-04-15T00:00:00+00:00 I am tired of re-installing gentoo (from an chroot approach) on my phone by connecting parts from different resources and so here is the guide to do it all in a go.</p> This guide assumes the prerequisite of a rooted phone. Just fire up an adb shell / termux and follow this :</em></p> First we proceed with downloading the correct stage3 tarballs</a>. Copy it in your phone's storage.</p> Then we make a folder for the Gentoo root. Since we are following SharkBait, we will use data/gnu</code> as root. Copy the stage3 into it.</p> mkdir</span> -</span>p</span> /data/gnu</span></span> cp</span> /sdcard/stage3-arm64-</span></span> /data/gnu/</span></span> cd</span> /data/gnu</span></span></code></pre> Then we extract the stage3</p> tar</span> xpvf</span> stage3-</span></span> `</span>--xattrs-include</span>=</span>'</span>.</span>'</span>`</span> --numeric-owner</span></span> #</span> no those backticks are not just a content generation error</span></span></code></pre> After this we will copy the DNS information in /etc/resolve.conf</code> so that networking works after entering the new environment.</p> cp</span> -</span>-dereference</span> /etc/resolv.conf</span> /data/gnu/etc/</span></span></code></pre> Mount the required filesystems:</p> cd</span> /</span></span> mount</span> -</span>t</span> proc</span> proc</span> /data/gnu/proc</span></span> busybox</span> mount</span> -</span>-rbind</span> /dev</span> /data/gnu/dev</span></span> busybox</span> mount</span> -</span>-rbind</span> /sys</span> /data/gnu/sys</span></span></code></pre> Finally we enter the new environment:</p> chroot</span> /data/gnu</span> /bin/bash</span></span> source</span> /etc/profile</span></span> export</span> PS1</span>=</span>"</span>(chroot) </span>$</span>{</span>PS1</span>}</span>"</span></span></code></pre> Add Portage to gid 3003(inet):</p> groupadd</span> -</span>g</span> 3003</span> inet</span> &&</span> gpasswd</span> -</span>a</span> portage</span> inet</span></span></code></pre>Configuring Portage and Emerging Packages:</h4> These for preventing mount errors while emerging:</p> mkdir</span> -</span>p</span> /dev/shm</span></span> mount</span> -</span>t</span> tmpfs</span> -</span>o</span> nodev,nosuid,noexec,mode=1777,size=6144m</span> tmpfs</span> /dev/shm</span></span> mount</span> /dev/pts</span> -</span>o</span> remount,gid=5,mode=</span>620</span></span></code></pre> Unless you want to stare at your phone for more than 4 hours, or just have a lot of time, I would recommend you to perform these steps while before going to bed, because this takes time. The best approach would be to do all this in a wake-lock(ed) termux session.</p> emerge-webrsync</span></span> emerge</span> -</span>-ask</span> -</span>-verbose</span> -</span>-update</span> -</span>-deep</span> -</span>-newuse</span> @world</span></span></code></pre>Configuring Timezone and Locales</h4> Set your timezone with the correct one from /usr/share/zoneinfo</code> :</p> echo</span> "</span>Asia/Calcutta</span>"</span> ></span> /etc/timezone</span></span> emerge</span> -</span>-config</span> sys-libs/timezone-data</span></span></code></pre> Configure locales by uncommenting the correct one from /etc/locale.gen</code>.</p> locale-gen</span></span> eselect</span> locale</span> list</span></span> #</span> proceed to chose the desired locale</span></span> env-update</span> &&</span> source</span> /etc/profile</span> &&</span> export</span> PS1</span>=</span>"</span>(chroot) </span>$</span>{</span>PS1</span>}</span>"</span></span></code></pre> I believe I have covered the information required until device specific steps. Now the part left is to add your device specific unofficial overlay, or continue with the official one, if your device is enlisted in it. An upcoming guide will guide you to make your kernel for SharkBait and finally, deploy SharkBait. Deploying SharkBait currently is a breeze for devices shipping with Android version lower than 9. Since mine did not, I would have to hack around the current script. If the results turn out to be positive, expect another blog coming up. Till then, Happy Hacking !</p>

Installing NixOS Remotely with nixos-anywhere

2025-04-10T00:00:00+00:00

nixos-anywhere</code> is a cool tool which can be used to installed nixos on:


an existing linux installation using kexec</li>
a baremetal machine</li>
and both of those options remotely using ssh</li>
</ul>
It is a part of the nix-community</a> suite,
and works well with
nixos-facter</a> for specifying
the hardware configuration and disko</a>
for declarative disk layout management.</p>
It is meant to run once and if successful you are left with your
desired NixOS configured according to your nixosConfigurations</code>.</p>
Setting up my physical homeserver</h2>
Let's set up the Dell Optiplex 3040 I am going to use as my homeserver. I'll
call this host mintaka</code>.</p>
The Optiplex was set to boot on BIOS by default, I changed that to UEFI
from the motherboard settings.</p>
Adding the host to the flake</h3>
Simply define the host in the flake.nix</p>
#</span> inside flake.nix</span></span>
hosts</span> =</span> {</span></span>
    #</span> ...</span></span>
</span>
    "</span>mintaka</span>"</span> =</span> lib</span>.</span>my</span>.</span>mkHostConfig</span> {</span></span>
      hostname</span> =</span> "</span>mintaka</span>"</span>;</span></span>
      system</span> =</span> "</span>x86_64-linux</span>"</span>;</span></span>
      username</span> =</span> "</span>wantguns</span>"</span>;</span></span>
      remoteBuild</span> =</span> true</span>;</span></span>
      ips</span> =</span> {</span></span>
        private</span> =</span> "</span>192.168.1.130</span>"</span>;</span></span>
      }</span>;</span></span>
    }</span>;</span></span>
}</span></span></code></pre>Booting</h3>
First, I created a bootable disk using an iso which supports
nixos-facter</code>, I am using
nixos-community/nixos-images</a></p>
And boot it on mintaka</code>, add my ssh key:</p>
mkdir</span> -</span>p</span> .ssh</span>;</span> echo</span> "</span>ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHcueIcj4fgzD6cUUGqituoupjexNNF1Hjrr+dyJ+gvA gunwant.jain@C02GH2V9MD6M.local</span>"</span> >></span> .ssh/authorized_keys</span></span></code></pre>nixos-facter</h3>
I am going to generate the hardware configuration for mintaka</code> using
nixos-facter:</p>
nixos-facter</span> ></span> hosts/linux/mintaka/facter.json</span></span></code></pre>
I'll add this copy of facter.json</code> to my nix flake.</p>
disko</h3>
Next, I'll declare my disk layout. I am choosing to try ZFS for the
first time. Other machines that I own run on BTRFS, and I haven't faced
any data loss as of now, but I also wanted to give OpenZFS a shot:</p>

  hosts/linux/mintaka/disk-config.nix</summary>
#</span> inside hosts/linux/mintaka/disk-config.nix</span></span>
{</span></span>
  disko</span>.</span>devices</span> =</span> {</span></span>
    disk</span> =</span> {</span></span>
      main</span> =</span> {</span></span>
        type</span> =</span> "</span>disk</span>"</span>;</span></span>
        device</span> =</span> "</span>/dev/sda</span>"</span>;</span></span>
        content</span> =</span> {</span></span>
          type</span> =</span> "</span>gpt</span>"</span>;</span></span>
          partitions</span> =</span> {</span></span>
            ESP</span> =</span> {</span></span>
              size</span> =</span> "</span>512M</span>"</span>;</span></span>
              type</span> =</span> "</span>EF00</span>"</span>;</span></span>
              content</span> =</span> {</span></span>
                type</span> =</span> "</span>filesystem</span>"</span>;</span></span>
                format</span> =</span> "</span>vfat</span>"</span>;</span></span>
                mountpoint</span> =</span> "</span>/boot</span>"</span>;</span></span>
                mountOptions</span> =</span> [</span> "</span>defaults</span>"</span> ]</span>;</span></span>
              }</span>;</span></span>
            }</span>;</span></span>
            zfs</span> =</span> {</span></span>
              size</span> =</span> "</span>100%</span>"</span>;</span></span>
              content</span> =</span> {</span></span>
                type</span> =</span> "</span>zfs</span>"</span>;</span></span>
                pool</span> =</span> "</span>zroot</span>"</span>;</span></span>
              }</span>;</span></span>
            }</span>;</span></span>
          }</span>;</span></span>
        }</span>;</span></span>
      }</span>;</span></span>
    }</span>;</span></span>
    zpool</span> =</span> {</span></span>
      zroot</span> =</span> {</span></span>
        type</span> =</span> "</span>zpool</span>"</span>;</span></span>
        rootFsOptions</span> =</span> {</span></span>
          mountpoint</span> =</span> "</span>none</span>"</span>;</span></span>
          compression</span> =</span> "</span>zstd</span>"</span>;</span></span>
          acltype</span> =</span> "</span>posixacl</span>"</span>;</span></span>
          xattr</span> =</span> "</span>sa</span>"</span>;</span></span>
          atime</span> =</span> "</span>off</span>"</span>;</span></span>
          relatime</span> =</span> "</span>on</span>"</span>;</span></span>
          "</span>com.sun:auto-snapshot</span>"</span> =</span> "</span>false</span>"</span>;</span></span>
        }</span>;</span></span>
        options</span> =</span> {</span></span>
          ashift</span> =</span> "</span>12</span>"</span>;</span></span>
          autotrim</span> =</span> "</span>on</span>"</span>;</span></span>
        }</span>;</span></span>
</span>
        datasets</span> =</span> {</span></span>
          "</span>root</span>"</span> =</span> {</span></span>
            type</span> =</span> "</span>zfs_fs</span>"</span>;</span></span>
            mountpoint</span> =</span> "</span>/</span>"</span>;</span></span>
          }</span>;</span></span>
          "</span>root/nix</span>"</span> =</span> {</span></span>
            type</span> =</span> "</span>zfs_fs</span>"</span>;</span></span>
            mountpoint</span> =</span> "</span>/nix</span>"</span>;</span></span>
          }</span>;</span></span>
          "</span>root/var</span>"</span> =</span> {</span></span>
            type</span> =</span> "</span>zfs_fs</span>"</span>;</span></span>
            mountpoint</span> =</span> "</span>/var</span>"</span>;</span></span>
          }</span>;</span></span>
          "</span>root/var/log</span>"</span> =</span> {</span></span>
            type</span> =</span> "</span>zfs_fs</span>"</span>;</span></span>
            mountpoint</span> =</span> "</span>/var/log</span>"</span>;</span></span>
          }</span>;</span></span>
          "</span>root/home</span>"</span> =</span> {</span></span>
            type</span> =</span> "</span>zfs_fs</span>"</span>;</span></span>
            mountpoint</span> =</span> "</span>/home</span>"</span>;</span></span>
          }</span>;</span></span>
          "</span>root/swap</span>"</span> =</span> {</span></span>
            type</span> =</span> "</span>zfs_volume</span>"</span>;</span></span>
            size</span> =</span> "</span>4G</span>"</span>;</span></span>
            content</span> =</span> {</span></span>
              type</span> =</span> "</span>swap</span>"</span>;</span></span>
            }</span>;</span></span>
            options</span> =</span> {</span></span>
              volblocksize</span> =</span> "</span>4096</span>"</span>;</span></span>
              compression</span> =</span> "</span>zle</span>"</span>;</span></span>
              logbias</span> =</span> "</span>throughput</span>"</span>;</span></span>
              sync</span> =</span> "</span>always</span>"</span>;</span></span>
              primarycache</span> =</span> "</span>metadata</span>"</span>;</span></span>
              secondarycache</span> =</span> "</span>none</span>"</span>;</span></span>
              "</span>com.sun:auto-snapshot</span>"</span> =</span> "</span>false</span>"</span>;</span></span>
            }</span>;</span></span>
          }</span>;</span></span>
        }</span>;</span></span>
      }</span>;</span></span>
    }</span>;</span></span>
  }</span>;</span></span>
}</span></span></code></pre></details>
I had already configured our nix flake to include the modules for disko
and our facter config with the nixosSystem builder in a previous blog 1</a></sup>.</p>
Installing</h3>
We can finally use the nix flake to install NixOS on mintaka</code>.
nixos-anywhere</code> looks for nixosConfiguration in the flake. I copied the
nix flake to the host and ran:</p>
nix</span> run</span> github:nix-community/nixos-anywhere</span> -</span>-</span> -</span>-flake</span> .#mintaka</span> -</span>-target-host</span> root@127.0.0.1</span> -</span>-build-on</span> remote</span></span></code></pre>
And after some time, the machine rebooted and I was presented with a
fresh OS install, with all my dotfiles and settings configured. For the
up-to-date summary of changes I have made on my host, I can now refer to
its git history</a></p>
Setting up my Oracle VM</h2>
Back when Oracle generously started handing out free instances, I
quickly got one in the Mumbai region. Oracle's choices for host
operating systems were limited and I went ahead with Ubuntu Linux at
that time.</p>
It is a meaty 4 arm64 core and 24GiB ram server, which I definetely want
to include in my fleet of servers. It also acted as test bed for remote
installtions and its the first and only arm64 machine in my fleet.</p>
I am naming this server bellatrix</code>. Let's get started.</p>
Facter</h3>
This VM was so old that facter failed to get some udev environment
variables which were added 2 years ago in systemd. I worked around this
by checking systemd's codebase history and posted my method on a
recently open github issue (now closed):

nix-community/nixos-facter #184</a></p>
Adding the host on my flake</h3>
I can finally bear the fruits of my labor, adding a new host is as easy
as adding a new commit:

wantguns/dotfiles: hosts: onboard bellatrix</a></p>
Here is the host attribute set entry:</p>
"</span>bellatrix</span>"</span> =</span> lib</span>.</span>my</span>.</span>mkHostConfig</span> {</span></span>
  hostname</span> =</span> "</span>bellatrix</span>"</span>;</span></span>
  system</span> =</span> "</span>aarch64-linux</span>"</span>;</span></span>
  username</span> =</span> "</span>wantguns</span>"</span>;</span></span>
  remoteBuild</span> =</span> true</span>;</span></span>
  ips</span> =</span> {</span></span>
    public</span> =</span> "</span><public-ip></span>"</span>;</span></span>
  }</span>;</span></span>
}</span>;</span></span></code></pre>
And this is disk config:</p>

  hosts/linux/bellatrix/disk-config.nix</summary>
#</span> Preserving the original fs layout</span></span>
{</span></span>
  disko</span>.</span>devices</span> =</span> {</span></span>
    disk</span> =</span> {</span></span>
      main</span> =</span> {</span></span>
        device</span> =</span> "</span>/dev/sda</span>"</span>;</span></span>
        type</span> =</span> "</span>disk</span>"</span>;</span></span>
        content</span> =</span> {</span></span>
          type</span> =</span> "</span>gpt</span>"</span>;</span></span>
          partitions</span> =</span> {</span></span>
            ESP</span> =</span> {</span></span>
              size</span> =</span> "</span>100M</span>"</span>;</span></span>
              type</span> =</span> "</span>EF00</span>"</span>;</span>  #</span> EFI System Partition</span></span>
              content</span> =</span> {</span></span>
                type</span> =</span> "</span>filesystem</span>"</span>;</span></span>
                format</span> =</span> "</span>vfat</span>"</span>;</span></span>
                mountpoint</span> =</span> "</span>/boot</span>"</span>;</span></span>
              }</span>;</span></span>
            }</span>;</span></span>
            root</span> =</span> {</span></span>
              size</span> =</span> "</span>100%</span>"</span>;</span>  #</span> Use all remaining space</span></span>
              content</span> =</span> {</span></span>
                type</span> =</span> "</span>filesystem</span>"</span>;</span></span>
                format</span> =</span> "</span>ext4</span>"</span>;</span></span>
                mountpoint</span> =</span> "</span>/</span>"</span>;</span></span>
                extraArgs</span> =</span> [</span>"</span>-L</span>"</span> "</span>cloudimg-rootfs</span>"</span>]</span>;</span>  #</span> Preserve the LABEL from fstab</span></span>
              }</span>;</span></span>
            }</span>;</span></span>
          }</span>;</span></span>
        }</span>;</span></span>
      }</span>;</span></span>
    }</span>;</span></span>
  }</span>;</span></span>
}</span></span></code></pre></details>
I decided to not tinker with the original disk layout a lot.</p>
Installing using kexec</h3>
With the host added and configured, I had to run a single command to
install NixOS through kexec</code> (thus overwriting the Ubuntu image):</p>
nix</span> run</span> github:nix-community/nixos-anywhere</span> -</span>-</span> -</span>-flake</span> .#bellatrix</span> -</span>-target-host</span> root@</span><</span>public-i</span>p</span>></span> -</span>-build-on</span> remote</span></span></code></pre>Concluding</h2>
Nix is helping me a lot to simplify operating a fleet of servers
declaratively. I can now make more changes to any of these servers by
running our deploy</code> nix flake app:</p>
nix</span> run</span> .#deploy</span> -</span>-</span> mintaka</span> true</span></span>
nix</span> run</span> .#deploy</span> -</span>-</span> bellatix</span> true</span></span></code></pre>
I can run this command from a remote machine which has SSH access to the
target machine, and it just works.</p>
Up next, I am going to try to create a common Wireguard
network between mintaka</code> and bellatrix</code> and try to create abstractions
which would make it easier for me to onboard a new Wireguard peer on my
fleet.</p>

^{1</sup>
https://wantguns.dev/nixmultihost/#lib-mkhost-nix</p>
</div>}

Single flake, multiple hosts

2025-03-14T00:00:00+00:00

I am done with setting up darwin machines now, and look forward to expand our nix files to declare state for linux hosts as well. The linux landscape is vast though, I could be logging into a work server, an embedded device, a server I rent, my personal workstation, etc. There is different amounts of state I want similar in these machines. And so it warrants for a better file structure.

.</span></span>
├── flake.lock</span></span>
├── flake.nix</span></span>
├── hosts</span></span>
│   ├── common</span></span>
│   │   └── default.nix</span></span>
│   ├── darwin</span></span>
│   │     ├── common.nix</span></span>
│   │     └── C02GH2V9MD6M</span></span>
│   │         ├── default.nix</span></span>
│   │         ├── git</span></span>
│   │         │   ├── ola</span></span>
│   │         │   └── olamessage</span></span>
│   │         └── home.nix</span></span>
│   └── linux</span></span>
│       ├── common.nix</span></span>
│       └── rigel</span></span>
│           ├── default.nix</span></span>
│           └── home.nix</span></span>
├── lib</span></span>
│   ├── default.nix</span></span>
│   ├── deploy.nix</span></span>
│   ├── mkHomeConfig.nix</span></span>
│   ├── mkHost.nix</span></span>
│   ├── mkHostConfig.nix</span></span>
│   └── utils.nix</span></span>
└──modules</span></span>
    └── features</span></span>
        ├── default.nix</span></span>
        ├── implementation.nix</span></span>
        ├── git</span></span>
        │   ├── config.nix</span></span>
        │   └── message</span></span>
        ├── nvim</span></span>
        │   ├── base.lua</span></span>
        │   └── treesitter.lua</span></span>
        ├── tmux</span></span>
        │   └── tmux.conf</span></span>
        └── zsh</span></span>
            ├── p10k.zsh</span></span>
            └── zshrc</span></span></code></pre>Abstractions</h2>
The idea is to basically have common sets of attributes at different
levels - per os and per host.</p>
Also, I don't frequently change my dotfiles, I have the opportunity to
abstract features like if I want neovim to be configured with LSP or
not, basically have a features attribute for home manager.</p>
modules/features/default.nix</h3>
I'll declare all the abstractions for home manager options in this file.</p>

  modules/features/default.nix</summary>
{</span> lib</span>,</span> ...</span> }</span>:</span></span>
</span>
with</span> lib</span>;</span></span>
</span>
{</span></span>
  options</span>.</span>features</span> =</span> {</span></span>
    editors</span> =</span> {</span></span>
      nvim</span> =</span> {</span></span>
        enable</span> =</span> mkEnableOption</span> "</span>neovim editor</span>"</span>;</span></span>
        ui</span> =</span> mkEnableOption</span> "</span>add ui elements</span>"</span>;</span></span>
        lsp</span> =</span> mkEnableOption</span> "</span>neovim LSP and treesitter support</span>"</span>;</span></span>
        #</span> ...</span></span>
      }</span>;</span></span>
    }</span>;</span></span>
</span>
    shell</span> =</span> {</span></span>
      zsh</span> =</span> {</span></span>
        enable</span> =</span> mkEnableOption</span> "</span>zsh shell</span>"</span>;</span></span>
        powerlevel10k</span> =</span> mkEnableOption</span> "</span>powerlevel10k theme</span>"</span>;</span></span>
      }</span>;</span></span>
    }</span>;</span></span>
</span>
    tmux</span> =</span> mkEnableOption</span> "</span>tmux terminal multiplexer</span>"</span>;</span></span>
</span>
    git</span> =</span> {</span></span>
        enable</span> =</span> mkEnableOption</span> "</span>git with delta</span>"</span>;</span></span>
        delta</span> =</span> mkEnableOption</span> "</span>git-delta</span>"</span>;</span></span>
    }</span>;</span></span>
  }</span>;</span></span>
}</span></span></code></pre></details>
modules/features/implementation.nix</h3>
I'll implement the declared options in this file.</p>

  modules/features/implementation.nix</summary>
{</span> config</span>,</span> pkgs</span>,</span> lib</span>,</span> ...</span> }</span>:</span></span>
</span>
let</span></span>
  cfg</span> =</span> config</span>.</span>features</span>;</span></span>
  </span></span>
  toLua</span> =</span> str</span>:</span> "</span>\n</span>lua << EOF</span>\n</span>${</span>str</span>}</span>\n</span>EOF</span>"</span>;</span></span>
  toLuaFile</span> =</span> file</span>:</span> "</span>\n</span>lua << EOF</span>\n</span>${</span>builtins</span>.</span>readFile</span> file</span>}</span>\n</span>EOF</span>"</span>;</span></span>
</span>
  fromGitHub</span> =</span> {</span> ref</span>,</span> repo</span>,</span> sha256</span> ?</span> lib</span>.</span>fakeSha256</span> }</span>:</span></span>
    pkgs</span>.</span>vimUtils</span>.</span>buildVimPlugin</span> {</span></span>
      pname</span> =</span> "</span>${</span>lib</span>.</span>strings</span>.</span>sanitizeDerivationName</span> repo</span>}</span>"</span>;</span></span>
      version</span> =</span> ref</span>;</span></span>
      src</span> =</span> pkgs</span>.</span>fetchFromGitHub</span> {</span></span>
        owner</span> =</span> lib</span>.</span>strings</span>.</span>elemAt</span> (</span>lib</span>.</span>strings</span>.</span>splitString</span> "</span>/</span>"</span> repo</span>)</span> 0</span>;</span></span>
        repo</span> =</span> lib</span>.</span>strings</span>.</span>elemAt</span> (</span>lib</span>.</span>strings</span>.</span>splitString</span> "</span>/</span>"</span> repo</span>)</span> 1</span>;</span></span>
        rev</span> =</span> ref</span>;</span></span>
        sha256</span> =</span> sha256</span>;</span></span>
      }</span>;</span></span>
    }</span>;</span></span>
</span>
  fakeVimPlugin</span> =</span> pkgs</span>.</span>runCommand</span> "</span>fakeVimPlugin</span>"</span> {</span> }</span> "</span>mkdir $out</span>"</span>;</span></span>
in</span></span>
{</span></span>
  config</span> =</span> lib</span>.</span>mkMerge</span> [</span></span>
    {</span></span>
      home</span> =</span> {</span></span>
        stateVersion</span> =</span> "</span>24.11</span>"</span>;</span></span>
        packages</span> =</span> with</span> pkgs</span>;</span> [</span></span>
          ripgrep</span></span>
        ]</span>;</span></span>
      }</span>;</span></span>
</span>
      programs</span>.</span>home-manager</span>.</span>enable</span> =</span> true</span>;</span></span>
    }</span></span>
</span>
    #</span> Neovim configuration</span></span>
    (</span>lib</span>.</span>mkIf</span> cfg</span>.</span>editors</span>.</span>nvim</span>.</span>enable</span> {</span></span>
      programs</span>.</span>neovim</span> =</span> {</span></span>
        enable</span> =</span> true</span>;</span></span>
        defaultEditor</span> =</span> true</span>;</span></span>
        viAlias</span> =</span> true</span>;</span></span>
        vimAlias</span> =</span> true</span>;</span></span>
        vimdiffAlias</span> =</span> true</span>;</span></span>
</span>
        plugins</span> =</span> with</span> pkgs</span>.</span>vimPlugins</span>;</span> [</span></span>
          #</span> Base configuration</span></span>
          {</span></span>
            plugin</span> =</span> fakeVimPlugin</span>;</span></span>
            config</span> =</span> toLuaFile</span> ./nvim/base.lua</span>;</span></span>
          }</span></span>
          plenary-nvim</span></span>
        ]</span> </span></span>
        </span></span>
        #</span> UI Elements - each plugin individually conditioned</span></span>
        ++</span> lib</span>.</span>optionals</span> cfg</span>.</span>editors</span>.</span>nvim</span>.</span>ui</span> [</span></span>
          {</span></span>
            plugin</span> =</span> fromGitHub</span> {</span></span>
              ref</span> =</span> "</span>HEAD</span>"</span>;</span></span>
              repo</span> =</span> "</span>bluz71/vim-moonfly-colors</span>"</span>;</span></span>
              sha256</span> =</span> "</span>c5fxaT8Bc5MzOjJsuU95K9yzTpGqj/7QP4GuLfsY4VE=</span>"</span>;</span></span>
            }</span>;</span></span>
            config</span> =</span> "</span>colorscheme moonfly</span>"</span>;</span></span>
          }</span></span>
        ]</span></span>
        </span></span>
        #</span> LSP and Treesitter</span></span>
        ++</span> lib</span>.</span>optionals</span> cfg</span>.</span>editors</span>.</span>nvim</span>.</span>lsp</span> [</span></span>
          {</span></span>
            plugin</span> =</span> nvim-treesitter</span>.</span>withAllGrammars</span>;</span></span>
            config</span> =</span> toLuaFile</span> ./nvim/treesitter.lua</span>;</span></span>
          }</span></span>
          {</span></span>
            plugin</span> =</span> nvim-lspconfig</span>;</span></span>
            config</span> =</span> toLuaFile</span> ./nvim/lsp.lua</span>;</span></span>
          }</span></span>
          {</span></span>
            plugin</span> =</span> nvim-cmp</span>;</span></span>
            config</span> =</span> toLuaFile</span> ./nvim/cmp.lua</span>;</span></span>
          }</span></span>
          cmp-nvim-lsp</span></span>
        ]</span></span>
      }</span>;</span></span>
    }</span>)</span></span>
</span>
    #</span> Shell configuration</span></span>
    (</span>lib</span>.</span>mkIf</span> cfg</span>.</span>shell</span>.</span>zsh</span>.</span>enable</span> {</span></span>
      programs</span>.</span>zsh</span> =</span> {</span></span>
        enable</span> =</span> true</span>;</span></span>
        syntaxHighlighting</span>.</span>enable</span> =</span> true</span>;</span></span>
        autosuggestion</span>.</span>enable</span> =</span> true</span>;</span></span>
        enableCompletion</span> =</span> true</span>;</span></span>
        defaultKeymap</span> =</span> "</span>viins</span>"</span>;</span></span>
</span>
        completionInit</span> =</span> ''</span>autoload -U compinit && compinit -u</span>''</span>;</span></span>
</span>
        history</span> =</span> {</span></span>
          append</span> =</span> true</span>;</span></span>
          extended</span> =</span> true</span>;</span></span>
          size</span> =</span> 10000</span>;</span></span>
          save</span> =</span> 1000000</span>;</span></span>
        }</span>;</span></span>
</span>
        initExtraFirst</span> =</span> builtins</span>.</span>readFile</span> ./zsh/zshrc</span>;</span></span>
</span>
        plugins</span> =</span> lib</span>.</span>optional</span> cfg</span>.</span>shell</span>.</span>zsh</span>.</span>powerlevel10k</span> {</span></span>
          name</span> =</span> "</span>zsh-powerlevel10k</span>"</span>;</span></span>
          src</span> =</span> "</span>${</span>pkgs</span>.</span>zsh-powerlevel10k</span>}</span>/share/zsh-powerlevel10k/</span>"</span>;</span></span>
          file</span> =</span> "</span>powerlevel10k.zsh-theme</span>"</span>;</span></span>
        }</span>;</span></span>
      }</span>;</span></span>
    }</span>)</span></span>
</span>
    #</span> Powerlevel10k configuration</span></span>
    (</span>lib</span>.</span>mkIf</span> (</span>cfg</span>.</span>shell</span>.</span>zsh</span>.</span>enable</span> &&</span> cfg</span>.</span>shell</span>.</span>zsh</span>.</span>powerlevel10k</span>)</span> {</span></span>
      home</span>.</span>file</span>.</span>"</span>.p10k.zsh</span>"</span>.</span>source</span> =</span> ./zsh/p10k.zsh</span>;</span></span>
    }</span>)</span></span>
</span>
    (</span>lib</span>.</span>mkIf</span> cfg</span>.</span>tmux</span> {</span></span>
      programs</span>.</span>tmux</span> =</span> {</span></span>
        enable</span> =</span> true</span>;</span></span>
        extraConfig</span> =</span> builtins</span>.</span>readFile</span> ./tmux/tmux.conf</span>;</span></span>
      }</span>;</span></span>
    }</span>)</span></span>
</span>
    (</span>lib</span>.</span>mkIf</span> cfg</span>.</span>git</span>.</span>enable</span> {</span></span>
      programs</span>.</span>git</span> =</span> {</span></span>
        enable</span> =</span> true</span>;</span></span>
        extraConfig</span> =</span> import</span> ./git/config.nix</span>;</span></span>
        delta</span>.</span>enable</span> =</span> cfg</span>.</span>git</span>.</span>delta</span>;</span></span>
      }</span>;</span></span>
</span>
      home</span>.</span>file</span>.</span>"</span>.config/git/message</span>"</span>.</span>source</span> =</span> ./git/message</span>;</span></span>
    }</span>)</span></span>
  ]</span>;</span></span>
}</span></span></code></pre></details>
This enables me to have a hosts/darwin/<hostname>/home.nix</code> like:</p>
{</span> config</span>,</span> pkgs</span>,</span> ...</span> }</span>:</span> {</span></span>
  features</span> =</span> {</span></span>
    editors</span>.</span>nvim</span> =</span> {</span></span>
      enable</span> =</span> true</span>;</span></span>
      ui</span> =</span> true</span>;</span></span>
      lsp</span> =</span> true</span>;</span></span>
    }</span>;</span></span>
</span>
    shell</span>.</span>zsh</span> =</span> {</span></span>
      enable</span> =</span> true</span>;</span></span>
      powerlevel10k</span> =</span> true</span>;</span></span>
    }</span>;</span></span>
</span>
    git</span> =</span> {</span></span>
      enable</span> =</span> true</span>;</span></span>
      delta</span> =</span> true</span>;</span></span>
    }</span>;</span></span>
</span>
    tmux</span> =</span> true</span>;</span></span>
  }</span>;</span></span>
</span>
  programs</span>.</span>git</span>.</span>includes</span> =</span> [</span> {</span> path</span> =</span> "</span>~/dev/.gitconfig</span>"</span>;</span> }</span> ]</span>;</span></span>
}</span></span></code></pre>
and the their system config, hosts/darwin/<hostname>/default.nix</code> like:</p>

  hosts/darwin/<hostname>/default.nix</summary>
#</span> Host-specific system configuration</span></span>
{</span> config</span>,</span> pkgs</span>,</span> ...</span> }</span>:</span> {</span></span>
  nix</span> =</span> {</span></span>
    settings</span> =</span> {</span></span>
      "</span>ssl-cert-file</span>"</span> =</span> "</span>/opt/nix-and-zscaler.crt</span>"</span>;</span></span>
    }</span>;</span></span>
  }</span>;</span></span>
</span>
  users</span>.</span>users</span>.</span>"</span><username></span>"</span> =</span> {</span></span>
    shell</span> =</span> pkgs</span>.</span>zsh</span>;</span></span>
    home</span> =</span> "</span>/Users/<username></span>"</span>;</span></span>
    packages</span> =</span> with</span> pkgs</span>;</span> [</span></span>
       xh</span></span>
       jq</span></span>
       tree</span></span>
    ]</span>;</span></span>
  }</span>;</span></span>
</span>
  homebrew</span> =</span> {</span></span>
    enable</span> =</span> true</span>;</span></span>
    brews</span> =</span> [</span></span>
     "</span>k9s</span>"</span></span>
     "</span>kubectl</span>"</span></span>
    ]</span>;</span></span>
  }</span>;</span></span>
}</span></span></code></pre></details>
Pretty simple right ? I can onboard new hosts and create similar "knobs"
to control state quickly.</p>
Implementation</h2>
In this series, we started with a standalone home-manager installation,
then pushed the home-manager configs into our nix-darwin config.</p>
A layman's definition for a nix flake would be that it accepts some
inputs and returns some outputs. Normally, these inputs are the source
to the definition of the packages you want in your outputs, and your
outputs are json like attribute sets which are parsed by a program like
nixos-rebuild or home-manager and do something with it, like building a
new nix profile.</p>
For my concerns, a nix flake that accepts home-manager, nix-darwin and
nixpkgs of course, and returns a set of homeConfigurations</code>,
darwinConfigurations</code> and nixosConfigurations</code> would make
everything easier. Because then I could then call home-manager,
darwin-rebuild or nix-rebuild on the same flake, which will become a
consolidated source of truth for all my hosts.</p>
Hosts attribute set</h3>
To declare the entrypoints and metadata for different hosts, lets
design a spec for hosts. Ideally I want to also be able to perform
remote builds</em> and remote deployments, thats possible for NAT'd hosts
if they are a part of my wireguard mesh. Basically we should declare a
set of reachable IPs (public, private) for every host. And of course the
username, hostname and the system.</p>
Something like this should work:</p>
hosts</span> =</span> {</span></span>
  "</span>C02GH2V9MD6M</span>"</span> =</span> {</span></span>
    hostname</span> =</span> "</span>C02GH2V9MD6M</span>"</span>;</span></span>
    system</span> =</span> "</span>x86_64-darwin</span>"</span>;</span></span>
    username</span> =</span> "</span>gunwant.jain1</span>"</span>;</span></span>
    justHome</span> =</span> false</span>;</span></span>
    remoteBuild</span> =</span> false</span>;</span></span>
  }</span>;</span></span>
</span>
  "</span>mintaka</span>"</span> =</span> {</span></span>
    hostname</span> =</span> "</span>mintaka</span>"</span>;</span></span>
    system</span> =</span> "</span>x86_64-linux</span>"</span>;</span></span>
    username</span> =</span> "</span>wantguns</span>"</span>;</span></span>
    remoteBuild</span> =</span> true</span>;</span></span>
    ips</span> =</span> {</span></span>
      private</span> =</span> "</span>192.168.1.5</span>"</span>;</span></span>
    }</span>;</span></span>
  }</span>;</span></span>
}</span>;</span></span></code></pre>
I added justHome</code> for letting the flake work on just
homeConfigurations, and remoteBuild</code> for building</em> the nix profile on
the remote host.</p>
Lets make a library function which defines this attribute for us:</p>
lib/mkHostConfig.nix</h3>

  lib/mkHostConfig.nix</summary>
#</span> inside ./lib/mkHostConfig.nix</span></span>
{</span> lib</span>,</span> ...</span> }</span>:</span></span>
</span>
let</span></span>
  utils</span> =</span> import</span> ./utils.nix</span> {</span> inherit</span> lib</span>;</span> }</span>;</span></span>
  inherit</span> (</span>utils</span>)</span> isDarwin</span> isLinux</span>;</span></span>
in</span></span>
{</span></span>
  mkHostConfig</span> =</span> config</span>:</span></span>
    let</span></span>
      platform</span> =</span></span>
        if</span> isDarwin</span> config</span>.</span>system</span> th</span>en</span> "</span>darwin</span>"</span></span>
        el</span>se</span> if</span> isLinux</span> config</span>.</span>system</span> th</span>en</span> "</span>linux</span>"</span></span>
        el</span>se</span> throw</span> "</span>Unsupported system: </span>${</span>config</span>.</span>system</span>}</span>"</span>;</span></span>
</span>
      defaults</span> =</span> {</span></span>
        remoteBuild</span> =</span> false</span>;</span></span>
        justHome</span> =</span> false</span>;</span></span>
        platform</span> =</span> platform</span>;</span></span>
      }</span>;</span></span>
    in</span></span>
    lib</span>.</span>recursiveUpdate</span> defaults</span> config</span>;</span></span>
}</span></span></code></pre></details>
I added a platform</code> field for the downstream libraries I will pass our
hosts</code> attribute into.</p>
lib/utils.nix</h3>
Here is the utils.nix</code> file:</p>

  lib/utils.nix</summary>
#</span> inside lib/utils.nix</span></span>
{</span> lib</span>,</span> ...</span> }</span>:</span></span>
</span>
let</span></span>
  isPlatform</span> =</span> system</span>:</span> platform</span>:</span> builtins</span>.</span>match</span> "</span>.*-</span>${</span>platform</span>}</span>"</span> system</span> !=</span> null</span>;</span></span>
  isDarwin</span> =</span> system</span>:</span> isPlatform</span> system</span> "</span>darwin</span>"</span>;</span></span>
  isLinux</span> =</span> system</span>:</span> isPlatform</span> system</span> "</span>linux</span>"</span>;</span></span>
in</span></span>
{</span></span>
  inherit</span> isPlatform</span> isDarwin</span> isLinux</span>;</span></span>
}</span></span></code></pre></details>
lib/mkHost.nix</h3>
Next, we'll make a library function that consumes this host attribute
set and returns a darwinConfiguration</code> or a nixosConfiguration</code>
depending on the system</code> and the platform</code> fields:</p>

  lib/mkHost.nix</summary>
#</span> inside lib/mkHost.nix</span></span>
{</span> lib</span>,</span> inputs</span>,</span> ...</span> }</span>:</span></span>
</span>
let</span></span>
  utils</span> =</span> import</span> ./utils.nix</span> {</span> inherit</span> lib</span>;</span> }</span>;</span></span>
  inherit</span> (</span>utils</span>)</span> isDarwin</span> isLinux</span>;</span></span>
in</span></span>
{</span></span>
  mkHost</span> =</span></span>
    hostConfig</span>@</span>{</span> hostname</span></span>
    ,</span> platform</span></span>
    ,</span> system</span></span>
    ,</span> username</span> ?</span> "</span>wantguns</span>"</span></span>
    ,</span> ...</span></span>
    }</span>:</span></span>
    let</span></span>
      hostDarwin</span> =</span> isDarwin</span> system</span>;</span></span>
      homeDirectory</span> =</span> if</span> hostDarwin</span> </span></span>
                      th</span>en</span> "</span>/Users/</span>${</span>username</span>}</span>"</span></span>
                      el</span>se</span> "</span>/home/</span>${</span>username</span>}</span>"</span>;</span></span>
</span>
      hmModule</span> =</span> if</span> hostDarwin</span></span>
                 th</span>en</span> inputs</span>.</span>home-manager</span>.</span>darwinModules</span>.</span>home-manager</span></span>
                 el</span>se</span> inputs</span>.</span>home-manager</span>.</span>nixosModules</span>.</span>home-manager</span>;</span></span>
</span>
      hostPath</span> =</span> ../hosts</span>/</span>${</span>platform</span>}</span>/</span>${</span>hostname</span>}</span>;</span></span>
      commonPath</span> =</span> ../hosts</span>/</span>${</span>platform</span>}</span>/common.nix</span>;</span></span>
      sharedPath</span> =</span> ../hosts/common/default.nix</span>;</span></span>
</span>
      diskoConfigPath</span> =</span> "</span>${</span>hostPath</span>}</span>/disk-config.nix</span>"</span>;</span></span>
      facterJsonPath</span> =</span> "</span>${</span>hostPath</span>}</span>/facter.json</span>"</span>;</span></span>
      hasDiskoConfig</span> =</span> !</span>hostDarwin</span> &&</span> builtins</span>.</span>pathExists</span> diskoConfigPath</span>;</span></span>
      hasFacterJson</span> =</span> !</span>hostDarwin</span> &&</span> builtins</span>.</span>pathExists</span> facterJsonPath</span>;</span></span>
</span>
      baseModules</span> =</span> [</span></span>
        sharedPath</span></span>
        commonPath</span></span>
        "</span>${</span>hostPath</span>}</span>/default.nix</span>"</span></span>
        hmModule</span></span>
      ]</span>;</span></span>
</span>
      linuxModules</span> =</span> if</span> hostDarwin</span> th</span>en</span> [</span>]</span> el</span>se</span></span>
        (</span>lib</span>.</span>optional</span> hasDiskoConfig</span> inputs</span>.</span>disko</span>.</span>nixosModules</span>.</span>disko</span>)</span> ++</span></span>
        (</span>lib</span>.</span>optional</span> hasDiskoConfig</span> diskoConfigPath</span>)</span> ++</span></span>
        (</span>lib</span>.</span>optional</span> hasFacterJson</span> inputs</span>.</span>nixos-facter-modules</span>.</span>nixosModules</span>.</span>facter</span>)</span> ++</span></span>
        (</span>lib</span>.</span>optional</span> hasFacterJson</span> {</span></span>
          facter</span>.</span>reportPath</span> =</span> facterJsonPath</span>;</span></span>
        }</span>)</span>;</span></span>
</span>
      systemBuilder</span> =</span> if</span> hostDarwin</span></span>
                      th</span>en</span> inputs</span>.</span>darwin</span>.</span>lib</span>.</span>darwinSystem</span></span>
                      el</span>se</span> inputs</span>.</span>nixpkgs</span>.</span>lib</span>.</span>nixosSystem</span>;</span></span>
    in</span></span>
    systemBuilder</span> {</span></span>
      inherit</span> system</span>;</span></span>
      specialArgs</span> =</span> {</span> inherit</span> inputs</span>;</span> }</span>;</span></span>
      modules</span> =</span> baseModules</span> ++</span> linuxModules</span> ++</span> [</span>{</span></span>
        nixpkgs</span> =</span> {</span></span>
          config</span>.</span>allowUnfree</span> =</span> true</span>;</span></span>
          hostPlatform</span> =</span> system</span>;</span></span>
        }</span>;</span></span>
</span>
        home-manager</span> =</span> {</span></span>
          useGlobalPkgs</span> =</span> true</span>;</span></span>
          useUserPackages</span> =</span> true</span>;</span></span>
          extraSpecialArgs</span> =</span> {</span> inherit</span> inputs</span>;</span> }</span>;</span></span>
          sharedModules</span> =</span> [</span></span>
            inputs</span>.</span>sops-nix</span>.</span>homeManagerModules</span>.</span>sops</span></span>
          ]</span>;</span></span>
          users</span>.</span>${</span>username</span>}</span> =</span> {</span> ...</span> }</span>:</span> {</span></span>
            imports</span> =</span> [</span></span>
              ../modules/features/default.nix</span></span>
              ../modules/features/implementation.nix</span></span>
              "</span>${</span>hostPath</span>}</span>/home.nix</span>"</span></span>
            ]</span>;</span></span>
          }</span>;</span></span>
        }</span>;</span></span>
      }</span>]</span>;</span></span>
    }</span>;</span></span>
}</span></span></code></pre></details>
You must be wondering what is disko</code> or facter</code> or sops</code>, and for
that, I'll be publishing a separate article. For now this config should
still work on darwin hosts, since all the linux related things are not
included if the system</code> is darwin.</p>
We are basically calling nixosSystem</code> or darwinSystem</code> and then using
the homeManager module specific to that system. Since many of our Home
Manager options are abstracted as à la carte features, we need to
include the features module we created.</p>
lib/mkHomeConfig.nix</h3>
Next lets create another library function to create
homeConfigurations</code>:</p>

  lib/mkHomeConfig.nix</summary>
#</span> inside lib/mkHomeConfig.nix</span></span>
{</span> lib</span>,</span> inputs</span>,</span> ...</span> }</span>:</span></span>
</span>
let</span></span>
  utils</span> =</span> import</span> ./utils.nix</span> {</span> inherit</span> lib</span>;</span> }</span>;</span></span>
  inherit</span> (</span>utils</span>)</span> isDarwin</span> isLinux</span>;</span></span>
in</span></span>
{</span></span>
  mkHomeConfig</span> =</span> hostname</span>:</span> hostConfig</span>:</span></span>
    let</span></span>
      system</span> =</span> hostConfig</span>.</span>system</span>;</span></span>
      username</span> =</span> hostConfig</span>.</span>username</span>;</span></span>
      platform</span> =</span> if</span> isDarwin</span> system</span> th</span>en</span> "</span>darwin</span>"</span> el</span>se</span> "</span>linux</span>"</span>;</span></span>
      hostPath</span> =</span> ../hosts</span>/</span>${</span>platform</span>}</span>/</span>${</span>hostname</span>}</span>;</span></span>
      homeDirectory</span> =</span></span>
        if</span> isDarwin</span> system</span></span>
        th</span>en</span> "</span>/Users/</span>${</span>username</span>}</span>"</span></span>
        el</span>se</span> "</span>/home/</span>${</span>username</span>}</span>"</span>;</span></span>
    in</span></span>
    inputs</span>.</span>home-manager</span>.</span>lib</span>.</span>homeManagerConfiguration</span> {</span></span>
      pkgs</span> =</span> inputs</span>.</span>nixpkgs</span>.</span>legacyPackages</span>.</span>${</span>system</span>}</span>;</span></span>
      extraSpecialArgs</span> =</span> {</span> inherit</span> inputs</span>;</span> }</span>;</span></span>
      modules</span> =</span> [</span></span>
        ../modules/features/default.nix</span></span>
        ../modules/features/implementation.nix</span></span>
        "</span>${</span>hostPath</span>}</span>/home.nix</span>"</span></span>
        {</span></span>
          home</span> =</span> {</span></span>
            inherit</span> username</span> homeDirectory</span>;</span></span>
          }</span>;</span></span>
        }</span></span>
      ]</span>;</span></span>
    }</span>;</span></span>
}</span></span></code></pre></details>
This should be pretty explanatory. You can go through my blogs on
home-manager</code> and nix-darwin</code> to have a better picture how our
abstraction is growing.</p>
lib/default.nix</h3>
We'll import all of our library functions through the default nix file in this directory.</p>

  lib/default.nix</summary>
#</span> inside lib/default.nix</span></span>
{</span> lib</span>,</span> inputs</span> }</span>:</span></span>
</span>
{</span></span>
  mkHost</span> =</span> (</span>import</span> ./mkHost.nix</span> {</span> inherit</span> lib</span> inputs</span>;</span> }</span>)</span>.</span>mkHost</span>;</span></span>
  mkHomeConfig</span> =</span> (</span>import</span> ./mkHomeConfig.nix</span> {</span> inherit</span> lib</span> inputs</span>;</span> }</span>)</span>.</span>mkHomeConfig</span>;</span></span>
  mkHostConfig</span> =</span> (</span>import</span> ./mkHostConfig.nix</span> {</span> inherit</span> lib</span>;</span> }</span>)</span>.</span>mkHostConfig</span>;</span></span>
  isPlatform</span> =</span> (</span>import</span> ./utils.nix</span> {</span> inherit</span> lib</span>;</span> }</span>)</span>.</span>isPlatform</span>;</span></span>
  isDarwin</span> =</span> (</span>import</span> ./utils.nix</span> {</span> inherit</span> lib</span>;</span> }</span>)</span>.</span>isDarwin</span>;</span></span>
  isLinux</span> =</span> (</span>import</span> ./utils.nix</span> {</span> inherit</span> lib</span>;</span> }</span>)</span>.</span>isLinux</span>;</span></span>
}</span></span></code></pre></details>
flake.nix</h3>
Finally, we have all the required nix code to iterate our flake:</p>

  flake.nix</summary>
{</span></span>
  description</span> =</span> "</span>Nix System Configurations</span>"</span>;</span></span>
</span>
  inputs</span> =</span> {</span></span>
    nixpkgs</span>.</span>url</span> =</span> "</span>github:nixos/nixpkgs/nixos-unstable</span>"</span>;</span></span>
    darwin</span>.</span>url</span> =</span> "</span>github:lnl7/nix-darwin/master</span>"</span>;</span></span>
    home-manager</span>.</span>url</span> =</span> "</span>github:nix-community/home-manager</span>"</span>;</span></span>
    disko</span>.</span>url</span> =</span> "</span>github:nix-community/disko/latest</span>"</span>;</span></span>
    nixos-facter-modules</span>.</span>url</span> =</span> "</span>github:numtide/nixos-facter-modules</span>"</span>;</span></span>
    sops-nix</span>.</span>url</span> =</span> "</span>github:Mic92/sops-nix</span>"</span>;</span></span>
</span>
    darwin</span>.</span>inputs</span>.</span>nixpkgs</span>.</span>follows</span> =</span> "</span>nixpkgs</span>"</span>;</span></span>
    home-manager</span>.</span>inputs</span>.</span>nixpkgs</span>.</span>follows</span> =</span> "</span>nixpkgs</span>"</span>;</span></span>
    disko</span>.</span>inputs</span>.</span>nixpkgs</span>.</span>follows</span> =</span> "</span>nixpkgs</span>"</span>;</span></span>
    sops-nix</span>.</span>inputs</span>.</span>nixpkgs</span>.</span>follows</span> =</span> "</span>nixpkgs</span>"</span>;</span></span>
  }</span>;</span></span>
</span>
  outputs</span> =</span> inputs</span>@</span>{</span> self</span>,</span> nixpkgs</span>,</span> darwin</span>,</span> home-manager</span>,</span> ...</span> }</span>:</span></span>
    let</span></span>
      lib</span> =</span> nixpkgs</span>.</span>lib</span>.</span>extend</span> (</span>final</span>:</span> prev</span>:</span> {</span></span>
        my</span> =</span> import</span> ./lib</span> {</span> inherit</span> inputs</span>;</span> lib</span> =</span> prev</span>;</span> }</span>;</span></span>
      }</span>)</span>;</span></span>
</span>
      systems</span> =</span> [</span> "</span>x86_64-linux</span>"</span> "</span>x86_64-darwin</span>"</span> "</span>aarch64-linux</span>"</span> "</span>aarch64-darwin</span>"</span> ]</span>;</span></span>
      forAllSystems</span> =</span> nixpkgs</span>.</span>lib</span>.</span>genAttrs</span> systems</span>;</span></span>
</span>
      hosts</span> =</span> {</span></span>
        "</span>C02GH2V9MD6M</span>"</span> =</span> lib</span>.</span>my</span>.</span>mkHostConfig</span> {</span></span>
          hostname</span> =</span> "</span>C02GH2V9MD6M</span>"</span>;</span></span>
          system</span> =</span> "</span>x86_64-darwin</span>"</span>;</span></span>
          username</span> =</span> "</span>gunwant.jain1</span>"</span>;</span></span>
          justHome</span> =</span> false</span>;</span></span>
          remoteBuild</span> =</span> false</span>;</span></span>
        }</span>;</span></span>
</span>
        "</span>mintaka</span>"</span> =</span> lib</span>.</span>my</span>.</span>mkHostConfig</span> {</span></span>
          hostname</span> =</span> "</span>mintaka</span>"</span>;</span></span>
          system</span> =</span> "</span>x86_64-linux</span>"</span>;</span></span>
          username</span> =</span> "</span>wantguns</span>"</span>;</span></span>
          remoteBuild</span> =</span> true</span>;</span></span>
          ips</span> =</span> {</span></span>
            private</span> =</span> "</span>192.168.1.5</span>"</span>;</span></span>
          }</span>;</span></span>
        }</span>;</span></span>
      }</span>;</span></span>
</span>
      forAllHosts</span> =</span> f</span>:</span> builtins</span>.</span>mapAttrs</span> f</span> hosts</span>;</span></span>
</span>
    in</span></span>
    {</span></span>
      inherit</span> hosts</span>;</span></span>
</span>
      darwinConfigurations</span> =</span> builtins</span>.</span>mapAttrs</span></span>
        (</span>name</span>:</span> hostConfig</span>:</span> lib</span>.</span>my</span>.</span>mkHost</span> hostConfig</span>)</span></span>
        (</span>lib</span>.</span>filterAttrs</span> (</span>name</span>:</span> hostConfig</span>:</span> lib</span>.</span>my</span>.</span>isDarwin</span> hostConfig</span>.</span>system</span>)</span> hosts</span>)</span>;</span></span>
</span>
      nixosConfigurations</span> =</span> builtins</span>.</span>mapAttrs</span></span>
        (</span>name</span>:</span> hostConfig</span>:</span> lib</span>.</span>my</span>.</span>mkHost</span> hostConfig</span>)</span></span>
        (</span>lib</span>.</span>filterAttrs</span> (</span>name</span>:</span> hostConfig</span>:</span> lib</span>.</span>my</span>.</span>isLinux</span> hostConfig</span>.</span>system</span>)</span> hosts</span>)</span>;</span></span>
</span>
      homeConfigurations</span> =</span> forAllHosts</span> lib</span>.</span>my</span>.</span>mkHomeConfig</span>;</span></span>
    }</span>;</span></span>
}</span></span></code></pre></details>
Simple enough, we can now use this flake for multiple tools:</p>
#</span> for home-manager</span></span>
nix</span> run</span> github:nix-community/home-manager/master</span> -</span>-</span> switch</span> -</span>-flake</span> .#</span>"</span>hostname</span>"</span></span>
</span>
#</span> for darwin-rebuild</span></span>
nix</span> run</span> github:lnl7/nix-darwin/master#darwin-rebuild</span> -</span>-</span> switch</span> -</span>-flake</span> .#</span>"</span>hostname</span>"</span></span>
</span>
#</span> for nixos-rebuild</span></span>
nix</span> run</span> nixpkgs#nixos-rebuild</span> -</span>-</span> switch</span> -</span>-flake</span> .#</span>"</span>hostname</span>"</span></span></code></pre>Remote deployment & Flake apps</h2>
I also want the ability to execute these commands remotely and push
changes remotely to all my hosts. There are a few tools which are
popular - deploy-rs</code>, colmena</code>, clan.lol</code>. I tried each of them and found
out that:</p>

Colmena</strong> does not support hosts of different systems in the same flake (issue</a>)</li>
Deploy-rs</strong> does not support building on remote</strong>, it only supports copying the build artifacts to remote (issue</a>)</li>
Clan.lol</strong> is too restrictive for my taste</li>
</ul>
I recently found out that nixos-rebuild has the remote build ability innately:</p>
nix</span> run</span> nixpkgs#nixos-rebuild</span> -</span>-</span> switch</span> \</span></span>
  -</span>-flake</span> .#</span>"</span>$</span>HOSTNAME</span>"</span> \</span></span>
  -</span>-build-host</span> "</span>$</span>USERNAME</span>@</span>$</span>IP</span>"</span> \</span></span>
  -</span>-target-host</span> "</span>$</span>USERNAME</span>@</span>$</span>IP</span>"</span> \</span></span>
  -</span>-use-remote-sudo</span> \</span></span>
  -</span>-fast</span> \</span></span>
  -</span>-use-substitutes</span> \</span></span>
  -</span>-option</span> builders-use-substitutes</span> true</span></span></code></pre>
this command will build the nixosConfiguration remotely and activate it
as well. This is great, we naturally reduce our dependencies for a
deployment tool. Now we can just create a new flake app</a> and call nix run</code> on our flake:</p>
#</span> inside flake.nix</span></span>
</span>
#</span> ... rest of the config</span></span>
      homeConfigurations</span> =</span> forAllHosts</span> lib</span>.</span>my</span>.</span>mkHomeConfig</span>;</span></span>
</span>
      apps</span> =</span> forAllSystems</span> (</span>system</span>:</span> {</span></span>
        deploy</span> =</span> {</span></span>
          type</span> =</span> "</span>app</span>"</span>;</span></span>
          program</span> =</span> "</span>${</span>import</span> ./lib/deploy.nix</span> {</span> </span></span>
            pkgs</span> =</span> nixpkgs</span>.</span>legacyPackages</span>.</span>${</span>system</span>}</span>;</span> </span></span>
            inherit</span> nixpkgs</span>;</span></span>
          }</span>}</span>/bin/deploy</span>"</span>;</span></span>
        }</span>;</span></span>
      }</span>)</span>;</span></span>
  }</span></span></code></pre>
and our deploy app is basically a shell script:</p>

  lib/deploy.nix</summary>
#</span> inside lib/deploy.nix</span></span>
{</span> pkgs</span>,</span> nixpkgs</span> }</span>:</span></span>
</span>
pkgs</span>.</span>writeShellScriptBin</span> "</span>deploy</span>"</span> ''</span></span>
  #!/usr/bin/env bash</span></span>
  set -e</span></span>
</span>
  RAW_HOSTNAME=</span>''$</span>{1:-$(hostname)}</span></span>
  HOSTNAME=$(echo "$RAW_HOSTNAME" | cut -d '.' -f 1)</span></span>
  REMOTE_BUILD=</span>''$</span>{2:-false}</span></span>
</span>
  SYSTEM=$(nix eval --raw .#hosts."$HOSTNAME".system)</span></span>
  USERNAME=$(nix eval --raw .#hosts."$HOSTNAME".username)</span></span>
  JUST_HOME=$(nix eval --json .#hosts."$HOSTNAME".justHome 2>/dev/null | grep -q "true" && echo "true" || echo "false")</span></span>
  REMOTE_BUILD_ENABLED=$(nix eval --json .#hosts."$HOSTNAME".remoteBuild | grep -q "true" && echo "true" || echo "false")</span></span>
</span>
  deploy_home() {</span></span>
    local host=$1</span></span>
    local remote=$2</span></span>
    echo "Deploying home configuration for $host..."</span></span>
  </span></span>
    if [ "$remote" == "true" ]; then</span></span>
      ssh "$USERNAME@$IP" "nix run 'github:nix-community/home-manager/master' -- switch --flake '.#$host'"</span></span>
    else</span></span>
      nix run github:nix-community/home-manager/master -- switch --flake .#"$host"</span></span>
    fi</span></span>
  }</span></span>
</span>
  if [[ "$REMOTE_BUILD" == "true" && "$REMOTE_BUILD_ENABLED" == "true" ]]; then</span></span>
    PUBLIC_IP=$(nix eval --raw .#hosts."$HOSTNAME".ips.public 2>/dev/null || echo "")</span></span>
    PRIVATE_IP=$(nix eval --raw .#hosts."$HOSTNAME".ips.private 2>/dev/null || echo "")</span></span>
  </span></span>
    IP="$PUBLIC_IP"</span></span>
    if ! ping -c 1 -W 1 "$IP" &>/dev/null && [ -n "$PRIVATE_IP" ]; then</span></span>
      IP="$PRIVATE_IP"</span></span>
    fi</span></span>
  </span></span>
    if [ -z "$IP" ]; then</span></span>
      echo "Error: No IP address available for remote build"</span></span>
      exit 1</span></span>
    fi</span></span>
  </span></span>
    if [[ "$JUST_HOME" == "true" ]]; then</span></span>
      deploy_home "$HOSTNAME" true</span></span>
    else</span></span>
      echo "Deploying to $HOSTNAME ($IP) with remote build..."</span></span>
    </span></span>
      if [[ "$SYSTEM" == *"-linux" ]]; then</span></span>
        nix run nixpkgs#nixos-rebuild -- switch \</span></span>
          --flake .#"$HOSTNAME" \</span></span>
          --build-host "$USERNAME@$IP" \</span></span>
          --target-host "$USERNAME@$IP" \</span></span>
          --use-remote-sudo \</span></span>
          --fast \</span></span>
          --use-substitutes \</span></span>
          --option builders-use-substitutes true</span></span>
      else</span></span>
        nix run github:lnl7/nix-darwin/master#darwin-rebuild -- switch \</span></span>
          --flake .#"$HOSTNAME" \</span></span>
          --build-host "$USERNAME@$IP" \</span></span>
          --target-host "$USERNAME@$IP" \</span></span>
          --fast</span></span>
      fi</span></span>
    fi</span></span>
  else</span></span>
    if [[ "$JUST_HOME" == "true" ]]; then</span></span>
      deploy_home "$HOSTNAME" false</span></span>
    else</span></span>
      echo "Deploying to $HOSTNAME locally..."</span></span>
    </span></span>
      if [[ "$SYSTEM" == *"-linux" ]]; then</span></span>
        sudo nix run nixpkgs#nixos-rebuild -- switch --flake .#"$HOSTNAME"</span></span>
      else</span></span>
        nix run github:lnl7/nix-darwin/master#darwin-rebuild -- switch --flake .#"$HOSTNAME"</span></span>
      fi</span></span>
    fi</span></span>
  fi</span></span>
''</span></span></code></pre></details>
Now we can run commands like:</p>
#</span> Deploy to current host</span></span>
nix</span> run</span> .#deploy</span></span>
</span>
#</span> Deploy to specific host</span></span>
nix</span> run</span> .#deploy</span> -</span>-</span> mintaka</span> </span></span>
</span>
#</span> Deploy to specific host with remote build</span></span>
nix</span> run</span> .#deploy</span> -</span>-</span> mintaka</span> true</span></span></code></pre>Future</h2>
I have uploaded these files on my github</a>. I
will keep tracking them and hopefully make abstractions for easier syncing if
necessary.</p>
Since we now have a good abstraction for our hosts, I am going to start
using this flake alongside nixos-anywhere and nixos-infect to start
installing nix on my servers. The plan is to onboard them and build a
wireguard mesh, and a K3S cluster. Starting with my Dell Optiplex 3040, mintaka</code>.</p>

Getting Started with Nix Home Manager on macOS

2025-03-06T00:00:00+00:00

I am going to start hacking nix code on my macbook first. Home Manager is a unobstrusive entry-point to the nix world. You need to first install nix with flakes. Use the determinate installer</a>.

Once thats done, create a new directory in your development folder, lets say nixhome</code>, and create a flake.nix</code>:


  flake.nix</summary>
{</span></span>
  description</span> =</span> "</span>Home Manager configuration</span>"</span>;</span></span>
</span>
  inputs</span> =</span> {</span></span>
    nixpkgs</span>.</span>url</span> =</span> "</span>github:nixos/nixpkgs/nixos-unstable</span>"</span>;</span></span>
    home-manager</span> =</span> {</span></span>
      url</span> =</span> "</span>github:nix-community/home-manager</span>"</span>;</span></span>
      inputs</span>.</span>nixpkgs</span>.</span>follows</span> =</span> "</span>nixpkgs</span>"</span>;</span></span>
    }</span>;</span></span>
  }</span>;</span></span>
</span>
  outputs</span> =</span> inputs</span>@</span>{</span> self</span>,</span> nixpkgs</span>,</span> home-manager</span>,</span> ...</span> }</span>:</span></span>
    let</span></span>
      system</span> =</span> "</span>x86_64-darwin</span>"</span>;</span></span>
      pkgs</span> =</span> nixpkgs</span>.</span>legacyPackages</span>.</span>${</span>system</span>}</span>;</span></span>
    in</span> {</span></span>
      homeConfigurations</span>.</span>"</span>play</span>"</span> =</span> home-manager</span>.</span>lib</span>.</span>homeManagerConfiguration</span> {</span></span>
        inherit</span> pkgs</span>;</span></span>
        modules</span> =</span> [</span></span>
          ./home.nix</span></span>
        ]</span>;</span></span>
      }</span>;</span></span>
    }</span>;</span></span>
}</span></span></code></pre></details>
and a home.nix</code> file:</p>

  home.nix</summary>
{</span> config</span>,</span> pkgs</span>,</span> lib</span>,</span> ...</span> }</span>:</span></span>
</span>
let</span></span>
  toLua</span> =</span> str</span>:</span> "</span>\n</span>lua << EOF</span>\n</span>${</span>str</span>}</span>\n</span>EOF</span>"</span>;</span></span>
  toLuaFile</span> =</span> file</span>:</span> "</span>\n</span>lua << EOF</span>\n</span>${</span>builtins</span>.</span>readFile</span> file</span>}</span>\n</span>EOF</span>"</span>;</span></span>
</span>
  fromGitHub</span> =</span></span>
    {</span></span>
      ref</span>,</span></span>
      repo</span>,</span></span>
      sha256</span> ?</span> lib</span>.</span>fakeSha256</span>,</span></span>
    }</span>:</span></span>
    pkgs</span>.</span>vimUtils</span>.</span>buildVimPlugin</span> {</span></span>
      pname</span> =</span> "</span>${</span>lib</span>.</span>strings</span>.</span>sanitizeDerivationName</span> repo</span>}</span>"</span>;</span></span>
      version</span> =</span> ref</span>;</span></span>
      src</span> =</span> pkgs</span>.</span>fetchFromGitHub</span> {</span></span>
        owner</span> =</span> lib</span>.</span>strings</span>.</span>elemAt</span> (</span>lib</span>.</span>strings</span>.</span>splitString</span> "</span>/</span>"</span> repo</span>)</span> 0</span>;</span></span>
        repo</span> =</span> lib</span>.</span>strings</span>.</span>elemAt</span> (</span>lib</span>.</span>strings</span>.</span>splitString</span> "</span>/</span>"</span> repo</span>)</span> 1</span>;</span></span>
        rev</span> =</span> ref</span>;</span></span>
        sha256</span> =</span> sha256</span>;</span></span>
      }</span>;</span></span>
    }</span>;</span></span>
</span>
  fakeVimPlugin</span> =</span> pkgs</span>.</span>runCommand</span> "</span>fakeVimPlugin</span>"</span> {</span> }</span> "</span>mkdir $out</span>"</span>;</span></span>
in</span></span>
</span>
{</span></span>
  nixpkgs</span> =</span> {</span></span>
    config</span> =</span> {</span></span>
      allowUnfree</span> =</span> true</span>;</span></span>
      allowUnfreePredicate</span> =</span> (</span>_</span>:</span> true</span>)</span>;</span></span>
    }</span>;</span></span>
  }</span>;</span></span>
</span>
  home</span>.</span>username</span> =</span> "</span><username></span>"</span>;</span></span>
  home</span>.</span>homeDirectory</span> =</span> "</span>/Users/<username></span>"</span>;</span></span>
</span>
  home</span>.</span>stateVersion</span> =</span> "</span>24.11</span>"</span>;</span></span>
  home</span>.</span>packages</span> =</span> [</span></span>
    pkgs</span>.</span>ripgrep</span></span>
    pkgs</span>.</span>neofetch</span></span>
  ]</span>;</span></span>
</span>
  home</span>.</span>file</span> =</span> {</span></span>
    "</span>.p10k.zsh</span>"</span>.</span>source</span> =</span> ./zsh/p10k.zsh</span>;</span></span>
    "</span>.config/git/message</span>"</span>.</span>source</span> =</span> ./git/message</span>;</span></span>
  }</span>;</span></span>
</span>
  fonts</span>.</span>fontconfig</span>.</span>enable</span> =</span> true</span>;</span></span>
  programs</span>.</span>home-manager</span>.</span>enable</span> =</span> true</span>;</span></span>
</span>
  programs</span>.</span>zsh</span> =</span> {</span></span>
    enable</span> =</span> true</span>;</span></span>
    syntaxHighlighting</span>.</span>enable</span> =</span> true</span>;</span></span>
    autosuggestion</span>.</span>enable</span> =</span> true</span>;</span></span>
    enableCompletion</span> =</span> true</span>;</span></span>
    defaultKeymap</span> =</span> "</span>viins</span>"</span>;</span></span>
</span>
    completionInit</span> =</span> ''</span>autoload -U compinit && compinit -u</span>''</span>;</span></span>
</span>
    history</span> =</span> {</span></span>
      append</span> =</span> true</span>;</span></span>
      extended</span> =</span> true</span>;</span></span>
      size</span> =</span> 10000</span>;</span></span>
      save</span> =</span> 1000000</span>;</span></span>
    }</span>;</span></span>
</span>
    initExtraFirst</span> =</span> builtins</span>.</span>readFile</span> ./zsh/zshrc</span>;</span></span>
</span>
    plugins</span> =</span> [</span></span>
      {</span></span>
        name</span> =</span> "</span>zsh-powerlevel10k</span>"</span>;</span></span>
        src</span> =</span> "</span>${</span>pkgs</span>.</span>zsh-powerlevel10k</span>}</span>/share/zsh-powerlevel10k/</span>"</span>;</span></span>
        file</span> =</span> "</span>powerlevel10k.zsh-theme</span>"</span>;</span></span>
      }</span></span>
    ]</span>;</span> </span></span>
  }</span>;</span></span>
</span>
  programs</span>.</span>tmux</span> =</span> {</span></span>
    enable</span> =</span> true</span>;</span></span>
    extraConfig</span> =</span> builtins</span>.</span>readFile</span> ./tmux/tmux.conf</span>;</span></span>
  }</span>;</span></span>
</span>
  programs</span>.</span>git</span> =</span> {</span></span>
    enable</span> =</span> true</span>;</span></span>
    extraConfig</span> =</span> builtins</span>.</span>readFile</span> ./git/config</span>;</span></span>
    delta</span>.</span>enable</span> =</span> true</span>;</span></span>
  }</span>;</span></span>
</span>
  programs</span>.</span>neovim</span> =</span> {</span></span>
    enable</span> =</span> true</span>;</span></span>
    defaultEditor</span> =</span> true</span>;</span></span>
    viAlias</span> =</span> true</span>;</span></span>
    vimAlias</span> =</span> true</span>;</span></span>
    vimdiffAlias</span> =</span> true</span>;</span></span>
</span>
    plugins</span> =</span> with</span> pkgs</span>.</span>vimPlugins</span>;</span> [</span></span>
</span>
      {</span></span>
        plugin</span> =</span> fakeVimPlugin</span>;</span></span>
        config</span> =</span> toLuaFile</span> ./nvim/base.lua</span>;</span></span>
      }</span></span>
</span>
      plenary-nvim</span></span>
      which-key-nvim</span></span>
      copilot-vim</span></span>
</span>
      {</span></span>
        plugin</span> =</span> nvim-treesitter</span>.</span>withAllGrammars</span>;</span></span>
        config</span> =</span> toLuaFile</span> ./nvim/treesitter.lua</span>;</span></span>
      }</span></span>
</span>
      {</span></span>
        plugin</span> =</span> (</span></span>
          fromGitHub</span> {</span></span>
            ref</span> =</span> "</span>HEAD</span>"</span>;</span></span>
            repo</span> =</span> "</span>bluz71/vim-moonfly-colors</span>"</span>;</span></span>
            sha256</span> =</span> "</span>c5fxaT8Bc5MzOjJsuU95K9yzTpGqj/7QP4GuLfsY4VE=</span>"</span>;</span></span>
          }</span></span>
        )</span>;</span></span>
        config</span> =</span> "</span>colorscheme moonfly</span>"</span>;</span></span>
      }</span></span>
</span>
      #</span> Redacted list of plugins ...</span></span>
    ]</span>;</span></span>
</span>
    extraPackages</span> =</span> with</span> pkgs</span>;</span> [</span></span>
      gopls</span></span>
      pyright</span></span>
    ]</span>;</span></span>
  }</span>;</span></span>
}</span></span></code></pre></details>
Neovim</h3>
In the home.nix</code> file, you can see that we defined three functions which
are used to activate three types of vim plugins:</p>

Ones which do not need any further configuration, like plenary</li>
Ones which do need a lua config to function well, like treesitter</li>
Ones which do not exist on the nixpkg store currently</li>
Also I wanted to append these configs in the order of the plugins, and
programs.neovim.extraconfig</code> added the config at the end of the .vim
config file. So i created a fakeVimPlugin and injected a base config in
order.</li>
</ul>
Other things</h3>
Home manager has a rich set of options to explore. I am currently using also
aerc and alacritty through it.</p>
File tree</h2>
This is the final state of my flake folder:</p>
├── flake.nix</span></span>
├── flake.lock</span></span>
├── git</span></span>
│   ├── config</span></span>
│   ├── message</span></span>
├── home.nix</span></span>
├── nvim</span></span>
│   ├── base.lua</span></span>
│   └── treesitter.lua</span></span>
├── tmux</span></span>
│   └── tmux.conf</span></span>
└── zsh</span></span>
    ├── p10k.zsh</span></span>
    └── zshrc</span></span></code></pre>Commands</h2>
To apply this config, you can either just build it with the flake as an
app or just use the binary in a nix-shell:</p>
home-manager</span> switch</span> -</span>-flake</span> .#play</span></span></code></pre>
Home manager maintains a list of generations and adds one whenever you
switch to a new nix output. You can easily rollback on switches.</p>
Future</h2>
Home manager is nice and unobtrusive piece of software which is fine for
setting up on foreign hosts. Let's move ahead and integrate home-manager
with nix-darwin</code> to also make system level changes.</p>

Setting up my homelab and cloud with Nix

2025-03-04T00:00:00+00:00

I have been maintaining a set of cloud servers, for the purpose of selfhosting a ton of software, VPNs, and random experimentation. Recently, I bought a Dell Optiplex 3040 for Rs. 5000 ($60), and I am planning to revamp all my scattered scripts and make orchestration easy for a one person team.

Tradeoffs</h3>
I have a pragramtic approach to selfhosting. Until now, I have only used traefik 2.0, docker-compose, caddy. I do have lots of services running and I wanted an orchestration solution which is flexible. Web apps are mostly running on cloud native technology nowadays, and so Kubernetes stood out easily.
As a selfhoster, I can foresee changes in infrastructure less often, and so I don't exactly want to treat my servers as cattle. I want to be able to audit everything and easily onboard new hosts though. Ansible is a good option, but it feels like a leaky abstraction for IaC. I am going to go yolo on nix and see how far it can take me.

Concepts</h3>
I have had multiple scattered thoughts on what designs fits best for my needs, and I have realised:

a terminal session is my forte, I want to be able to replicate my dotfiles on new hosts.</li>
I am fine with running NixOS as my userland. I can also run nix-darwin on Macbooks.</li>
I am going to hand over all the hardware and os-level configuration to Nix, and will use a deployment tool like deploy-rs.</li>
I want all of my nodes to form a mesh, preferrably using wireguard, and preferrably not using proprietary tools like Tailscale, need to make a tradeoff for that.</li>
I want to orchestrate selfhosted services over different nodes, with affinities and selectors and resource constraints. I'll use Kubernetes, or K3s, more accurately to acheive this.</li>
I also like Gitops, mostly for the auditing and single source of truth features. Planning to use FluxCD and Github for gitops.</li> </ul>
With a setup like this, I will be able to setup new servers and handle any sort of load pretty quickly. The good thing with nix is that we can also have tiers based on how much control I want to give to new hardware, I might just want my normal terminal session on a contract work's server.
Dev logs</h3>
I am going to start blogging about my journey, here is timeline i'll keep updating:

Home Manager</a></li>
Nix Darwin</a></li>
Single Flake Multiple Hosts</a>

files available on github</a></li> </ul> </li>
Setting up Mintaka and Bellatrix</a></li>
Setting up a Wireguard network</li> </ul>

Building LMDB JNI for Jetson Nano

2022-02-20T00:00:00+00:00

Building</h2>
Once again, I hate to use docker for automating VM purposes, but I acknowledge it as an ad-hoc method for this instance.
We will be compiling natively for arm64 in a qemu image which is running inside a docker container.
# Import info for 64-bit Qemu based build FROM balenalib/raspberrypi4-64-python:latest-bullseye-build ARG LMDB_REPO=https://github.com/deephacks/lmdbjni ARG LMDB_REPO_BRANCH=master # Enforces cross-compilation through Qemu. RUN [ "cross-build-start" ] RUN sudo apt update -y && sudo apt upgrade -y RUN sudo apt install -y \ build-essential \ git \ tar \ default-jdk \ automake \ libtool \ maven \ neovim # Prepare LMDB-JNI Repo WORKDIR /code RUN git clone --single-branch --branch ${LMDB_REPO_BRANCH} --recursive ${LMDB_REPO} lmdb # Start build RUN cd lmdb && \ sed pom.xml -i -e "s/<hawtjni-version>.<\/hawtjni-version>/<hawtjni-version>1.15<\/hawtjni-version>/" && \ mvn clean install -P linux64 -Dhttps.protocols=TLSv1.2 # Build Output RUN find . | grep .jar VOLUME ["./artifacts"] RUN [ "cross-build-end" ]%</code></pre>Pitfalls</h2>
SSL</h3>
`SSL peer shut down incorrectly</code></pre> Fix: use the -Dhttps.protocols=TLSv1.2</code> flag when invoking maven.`
`Hawtjni</h3>`[INFO] | ^~~~~~ [INFO] src/hawtjni-callback.c:479:3: error: initializer element is not constant [INFO] 479 | (jintLong)FN(127, args), \ [INFO] | ^ [INFO] src/hawtjni-callback.c:498:3: note: in expansion of macro ‘FN_A_BLOCK’ [INFO] 498 | FN_A_BLOCK(12) [INFO] | ^~ [INFO] src/hawtjni-callback.c:479:3: note: (near initialization for ‘fnx_array[12][127]’) [INFO] 479 | (jintLong)FN(127, args), \ [INFO] | ^ [INFO] src/hawtjni-callback.c:498:3: note: in expansion of macro ‘FN_A_BLOCK’ [INFO] 498 | FN_A_BLOCK(12) [INFO] | ^~~~~~ [INFO] make: [Makefile:410: hawtjni-callback.lo] Error 1 [INFO] rc: 2 [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary for lmdbjni-project 0.4.8-SNAPSHOT: [INFO] [INFO] lmdbjni-project .................................... SUCCESS [02:08 min] [INFO] lmdbjni ............................................ SUCCESS [04:50 min] [INFO] lmdbjni-linux64 .................................... FAILURE [01:50 min] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 08:49 min [INFO] Finished at: 2022-02-21T09:40:07Z [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.fusesource.hawtjni:maven-hawtjni-plugin:1.11:build (default) on project lmdbjni-linux64: build failed: org.apache.maven.plugin.MojoExecutionException: make based build failed with exit code: 2 -> [Help 1]</code></pre> The error arises due to hardcoding a macro defining the system architecture to __x86_64__</code> instead of using something more generic. This pull request</a> fixes the bug, but it got merged into hawtjni v1.15</code> and LMDB-jni uses hawtjni v1.11</code>. Fix Apply basic substitution1</a>: sed pom.xml -i -e "s/<hawtjni-version>.*<\/hawtjni-version>/<hawtjni-version>1.15<\/hawtjni-version>/"</code></pre> ^1We could have used a more recent version, but org.fusesource.hawtjni</a> changed the package name after v1.15</code></a> and would therefore require more work. tldr; i am lazy. </div>

Final Report - GSoC 2020

2020-08-26T00:00:00+00:00

Its almost 3 months since I have been officially working with the SharkBait team at Gentoo. A lot of ground has been covered, a lot is still left. In this blog, I will explain the scope of SharkBait right now and how I plan its future.

Work Covered</h2>

Porting SharkBaitOS</h3>
When KireinaHoro started working on SharkBait, the latest Android version of that time was 8 or Android Oreo. Sharkbait at that time, supported all the Android devices because it targeted only a specific boot order, which was common till Android 8. Then came Android 9, with its System-As-Root configuration, which rendered SharkBait's original boot order non functional for all devices shipping with Android 9.
The first month of my work revolved around porting SharkBait to SAR based devices. I succeeded in doing so, and wrote blogs about it:

Android Boot High Jinks</a>
In this blog we discuss the different approaches at booting Android, Google has experimented with.</li>
Booting Gentoo from SAR-Preinit</a>
This blog showcases how I patched my phone's kernel in order to boot Gentoo on my phone.</li>
Starting System-as-Root Android in a LXC container</a>
This blog is the first class guide for launching SharkBait in System-As-Root devices.</li> </ul>
The projects made in efforts to port SharkBait for SAR were:

SAR-Preinit</a>
This repository acts as the setup which patches your custom kernel and makes it boot a GNU/Linux distro, instead of Android on your phone.</li>
Bootstrap-init</a>
This repository hosts the source of the init binary executed inside the LXC container which launches System-As-Root Android on top of Gentoo.</li> </ul> Introducing Android builds on AArch64 Hosts</h3>
As simple as it sounds, this part of my GSoC tenure turned out to be the exact opposite.
My first task was to provide a AArch64 native port of the LLVM toolchain AOSP uses. I was stuck on compiling and testing my modified scripts till the second month, when my mentor generously set up a beefy AArch64 server for me. I succeeded in delivering that toolchain. Related sources are:

Building Android: BTS</a>
In this blog, I look what events get triggered under the hood when we build Android, as to seek insight on breaking down Android later.</li>
Toolchain_LLVM_Android</a>
This repository hosts the build scripts and instructions on building AOSP's LLVM distribution on AArch64 Host, which targets AArch64 and Arm platforms</li> </ul>
Once I dealt with the toolchains, I could focus on building Android. Unfortunately this part is still unfinished due to the massive scale of thought-process and work it requires. Still, I will report on whatever I have been doing:

Manifest</a>
This is the (yet-to-be) SharkBait Manifest. It houses the different AOSP components SharkBait pulls in order to build Android. Some of which I have been working on:
android_build_blueprint</a></li>
android_build_soong</a></li>
android_build</a></li> </ul> </li> </ul>
Future Work</h2>
There are many things to be concerned with such as the flow of project development to follow, how exactly we should aim introducing arm64 as an host.
Plus the final goal of SharkBait is the ability to upgrade its Android container with the use of Portage, thus providing granular, GNU like updates to Android, quite possibly the first time ever. Keeping all this in mind we needed a proper roadmap on how to achieve this. My vision for it resembles somewhat like the following:

Build Android as an whole separate entity on AArch64.</li>
Then compose a monolithic ebuild for the exact same thing.</li>
Start breaking parts from Android:

Ship the toolchains used by AOSP separately using Portage and possibly integrate it with crossdev.</li>
Ship the necessary prebuilt build-tools</a> AOSP bundles as packages with Portage on arm64.</li> </ul> </li>
Capitalizing on the fact that AOSP code is highly modular, separate those modules into different packages under a common overlay.</li> </ol>
And since I consider myself apt in bringing this for SharkBait, I would love to work on it even after GSoC finishes.
Acknowledgements</h2>
I immensely appreciate the patience of my mentor, Benda Xu. He has always given me the helm when it comes to SharkBait, while still actively monitoring my work and helping me out. Without his generous efforts of providing me with an industry-standard arm64 server, I dont think I could consider this GSoC a success.
My co-mentors - Pengcheng Xu, Lucas Ramage and Stephen Christie helped me whenever I got stuck in my work. Pengcheng offered me assistance in getting UART work for my Phone, Lucas actively kept a lookout on my work and Stephen would offer me assistance in getting better hardware without two thoughts.
All of them offered crucial personal help whenever I was stuck, be it my work, understanding the project or even my proposal writing, something which I can never overlook.
The least I can do to return their favours is to bring SharkBait to all its glory, which I eagerly wait for.

Building Android: BTS

2020-07-13T00:00:00+00:00

So you've successfully compiled AOSP. But you are still not entertained and rather inquisitive about how things worked under the hood after you hit make bacon</code> or brunch</code> on your Lineage tree.

I am too and so I decided to take a look under the myriad that AOSP is. In this blog, I would share what I have learned so far, also being relevent to my GSoC obligations and therefore prepping hand by hand for a native aarch64 build.

So just setup your build environment and follow along.

Toolchains</h2>
Android has decided to phase out of using a GCC toolchain to the
LLVM/Clang based solution since Android 8.0. People usually see a
performance boost after the switch.</p>
The actual Toolchain AOSP uses is already prebuilt and bundled with the
entire AOSP source. But since AOSP only supports a Linux/Darwin x86 host
enviroment for building, we will have to build our own toolchain
supporting aarch64 hosts.</p>
In order to do that we shall dive deep first and study llvm_android</a>,
the source of the prebuilt toolchain.</p>
LLVM_android</h3>
I should mention that we could have used vanilla LLVM, but Google pushes some patches</a> along with its LLVM fork which is required by AOSP. So it is better if we stick to the way Google handles it. These patches are applied to the toolchain in between its compilation.</em></p>
After setting that toolchain up and doing a build test on x86, we
realise that LLVM_android ships with prebuilt GCC and Clang toolchains.
It uses Clang as its compiler and GCC's libs and binaries to build the
stage 1 of the LLVM_android compiler.</p>
LLVM_android also ships with the Bionic (Android's libc) sources. Though we notice while building the toolchain that only Bionic's headers are copied to the building toolchain. Bionic is not compiled at this stage.</p>
After building swig</code> and libedit</code> we finally reach the start of stage
2.
As expected the script now calls CMake again but with debug flags
referring to the newly built Clang/++ and its libraries. It also calls
the prebuilt libraries like it did earlier.</p>
Briefly put these are all the building parts, in order:</p>

For Linux:

Stage 1:  builds the crude Clang</li>
swig:     required for Stage 2</li>
libedit:  required for Stage 2</li>
Stage 2:  builds the final Clang</li>
</ul>
</li>
For Android { arm, aarch64, x86, i386 }:

sysroots</li>
compiler-rt</li>
compiler-rt-i386-host (only for linux)</li>
libomp</li>
lldb-server</li>
asan-mapfile</li>
</ul>
</li>
</ul>
We will have to patch paths.py</a> and replace the paths where the script calls its tools with the host's to compensate for the aarch64 host.
Additionally, we should only sync</code> the repositories which are necessary by editing the .repo/manifest.xml</code>.</p>
But enough of that, let's move on to building Android now.</p>
Building Android using the toolchain we just made</h2>
If not done already, sync Lineage's Android Manifest.
Put the newly made toolchain at prebuilts/clang/host/linux-x86/</code>
and change the Clang version to that of the one we just built at
build/soong/cmd/cc/config/global.go</code>.</p>
Now setup the environment and brunch your device.</p>
source</span> build/envsetup.sh</span></span>
brunch</span> _TARGET_NAME_</span></span></code></pre>
You'll notice that Building would give errors due to warnings.
We come to yet another realisation that AOSP uses an older version of Clang (9.0.3 currently).
The toolchain we just built is at bleeding edge, and is throwing errors at the old code as it
finds vulnerabilities in it.
We should recall that the lineage tree we synced is stable and so all
the warnings generated by the newer compiler can actually be avoided. We
could approach this in two ways:</p>

Disable certain flags which throws those errors.</li>
compile the current supported version of clang and use it instead.</li>
</ul>
Approach 1 would be time intensive and not scalable in any way, since we
don't control the main tree and upstream could make our forks fatal.
So we should follow the second approach.</p>
And so once we get a hold of the correct toolchain version, the builds
should not be throwing errors.</p>
Let me explain the build process now. Once you make bacon</code> or brunch</code>,
the soong build system is called and is compiled.
The very first program built is soong_ui</code> which is the build
system used to iniate the build process. The target you lunch will be
passed to the newly built soong_ui</code>.</p>
Go is used to build all of soong</code>, and AOSP ships its prebuilt version of Go.
We will have to change the location of GOROOT to use host's tools.
GOROOT is called at build/blueprint/microfactory/microfactory.bash</code>
so we will edit it there.</p>
Eventually it parses all the blueprint files (.bp extensions), and kati parses all
the makefiles. Once the build.ninja</code> file is created, the Android build
is ready to start.</p>
build.ninja</code> is called and it starts building certain intermediates
first which would be used later in assistance to build Android. One of
these intermediates which is of concern to us is Bionic</code></strong>.</p>
Compiling Bionic</h3>
Like we discussed earlier, Bionic's header files are already included in
the llvm_android toolchain. But the libc</code> sources are not compiled then.</p>
libc</code> and friends are actually compiled within the build as
intermediates and then copied to some different folder under /out</code>, from where it
gets called later in the build process.</p>
So, as I believe, once we have the toolchain ready, we should be good to
go for building Android using aarch64 host. I will be setting up an
ARM based VPS for the same, since I only have 32 gig of storage on my
phone.</p>

Conclusion</h2>
We need to build llvm_android on aarch64 host, by patching the build scripts provided by Google.
Also we need to build the latest supported version</a> of Clang/LLVM.
Once that is done, we will have to edit the paths of various tools called in the AOSP tree with our host's. And then finally, test a build.</p>
Achieving this should mark my task of building Android on aarch64 Host,
which is incomplete in the sense that I don't have a separatable
toolchain. But once we are through with this process, Separating Bionic
and implementing crossdev for the toolchain would be easier.</p>

Starting System-as-Root Android in a LXC container

2020-06-23T00:00:00+00:00

As I progress in my GSoC journey, my next task is to boot System-As-Root based Android in an LXC container inside Gentoo. As the name suggests, SAR devices use /system as their rootdir instead of /boot. SharkBait was based on the older booting mechanism. If you would want to know more about Android's Booting mechanism, I would recommend to have a look at this blog</a>.

Basically if you own a device that shipped with Android 9 (even the ones which are updated to Android 10) you would want to follow this guide.

Prerequisites</h2>
This setup expects some prerequisites which are as follows:

Have Gentoo installed as a chroot in your Android device. guide</a></li>
You should have a LXC enabled Kernel. guide</a></li>
You should have encryption disabled on your phone. guide</a></li>
You should have SELinux disabled.</li>
You should have the schedtune cgroup disabled.</li> </ol>
The 4th and 5th prerequisites are addressed in the next section.
Preparing the boot image</h2>
Clone the SAR-Preinit repository
git clone https://gitlab.com/WantGuns/sar-preinit.git --depth 1</code></pre> Pull out your device's boot.img</code> in the artifacts directory.
cd sar-preinit adb root adb shell dd if=/dev/block/bootdevice/by-name/boot | dd of=artifacts/boot.img</code></pre> Now unpack the boot.img using the Makefile provided and edit the bootimg.cfg file produced in out/unpack</code> directory.
make unpack nvim out/unpack/bootimg.cfg</code></pre> Add this string to the cmdline</code> key, if not already present.
androidboot.selinux=permissive cgroup_disable=schedtune</code></pre> This addresses the 4th and 5th prerequisites. If the kernel you are currently using is not LXC-enabled, you could swap the zImage inside the unpack directory with the LXC-enabled kernel image. Now we can finally repack and patch the boot image for the use of SharkBait.make repack</code></pre> The boot-mod.img</code> under the out directory is the patched boot which we need. Just don't flash it yet. Preparing the bootstrap-init</h2> Since System-As-Root requires a special environment unlike Android's older iterations, we need a init to bootstrap Android in the container. Currently the bootstrap-init requires NDK for building, this is subject to change for native compilation later. People who are in a hurry could checkout the pre-built binaries provided here</a> and skip the hassle of manually setting up their environment and compiling it. If you're not one of them, continue following. Clone the repository and edit the Makefile's provided NDK path with your own NDK path. git clone https://gitlab.com/WantGuns/bootstrap-init.git --depth 1 cd bootstrap-init nvim Makefile</code></pre> Then call make</code> in the bootstrap-init folder. This will build the required binary for different architechtures, we should focus on arm64-v8a inside the libs folder. Copy the downloaded/compiled bootstrap-init to Gentoo's home, we will need it later. adb push libs/arm64-v8a/bootstrap-init /data/gnu/home/init</code></pre>Preparing Gentoo for Android</h2> Chroot into Gentoo and start setting it up. adb root adb shell ## Now switched to Android's shell # Generate SSH key pairs, (will come in use later) ssh-keygen -t ed25519 -C android -f /data/ssh/id_ed25519 mount -t proc /proc /data/gnu/proc mount --rbind /dev /data/gnu/dev mount --rbind /sys /data/gnu/sys chroot /data/gnu /bin/bash ## Now switched to Gentoo's root source /etc/profile mkdir -p /var/lib/android/{system,vendor,data,system,cache} mkdir -p /var/lib/lxc/android/{rootfs,run,artifacts}</code></pre> All the commands henceforth are executed in the Gentoo chroot unless specified otherwise. Fstab</h3> Edit Gentoo's fstab with mount entries for Android's partitions. ####/etc/fstab##### # Android mounts /dev/mmcblk0p63 /var/lib/android/system ext4 ro,barrier=1,inode_readahead_blks=8 0 0 /dev/mmcblk0p55 /var/lib/android/persist ext4 noatime,nosuid,nodev,barrier=1,data=ordered,nomblk_io_submit 0 0 /dev/mmcblk0p64 /var/lib/android/vendor ext4 ro,barrier=1,inode_readahead_blks=8 0 0 /dev/mmcblk0p62 /var/lib/android/cache ext4 noatime,nosuid,nodev,barrier=1,data=ordered,nomblk_io_submit,noauto_da_alloc 0 0 /dev/mmcblk0p66 /var/lib/android/data ext4 noatime,nosuid,nodev,barrier=1,data=ordered,nomblk_io_submit,noauto_da_alloc,inode_readahead_blks=8 0 0 # Bind into container /run /var/lib/lxc/android/rootfs/run none bind 0 0</code></pre>Install LXC</h3> At the time of writing this blog, LXC's version 4 does not work for the setup. I would recommend to install the version 3. emerge -av --autounmask=y =app-emulation/lxc-3.0.3</code></pre> After this is done, setup the helper scripts for the container. cd /var/lib/lxc/android ## lxc's config cat > config <<EOF lxc.uts.name = android lxc.rootfs.path = /var/lib/lxc/android/rootfs/ lxc.init.cmd = /init lxc.net.0.type = none lxc.autodev = 0 lxc.log.level = 0 lxc.log.file = /var/log/lxc/hellalxc.log lxc.hook.pre-start = /var/lib/lxc/android/pre-start.sh lxc.hook.post-stop = /var/lib/lxc/android/post-stop.sh lxc.cgroup.cpu.rt_runtime_us = 950000 lxc.console.path = none EOF ## pre-start.sh cat > pre-start.sh <<EOF #!/bin/sh cp /var/lib/lxc/android/artifacts/init /var/lib/lxc/android/rootfs/init mkdir -p /var/lib/lxc/android/rootfs/dev mkdir -p /run if [ -f /run/.android-shutdown ]; then # we're shutting down; don't respawn echo "Shutting down, won't restart." exit 1 fi # handle cgroup mkdir /sys/fs/cgroup/cpu/lxc echo 950000 > /sys/fs/cgroup/cpu/lxc/cpu.rt_runtime_us EOF ## post-stop.sh cat > post-stop.sh <<EOF #!/bin/bash # do not reboot / restart when debugging if [ -f /run/.android-debug ]; then echo "Debug mode, won't power off." exit 0 fi # clean rootfs for SAR-enabled devices. rm -rf /var/lib/lxc/android/rootfs/* # leave mark to prevent container restart touch /run/.android-shutdown if [ -f /run/.android-reboot ]; then echo "Rebooting..." reboot else echo "Powering off..." poweroff fi EOF</code></pre> Copy the bootstrap-init binary to the artifacts folder. cp /home/init /var/lib/lxc/android/artifacts/init</code></pre> If it helps, this is the /var/lib/lxc/android</code> tree : localhost /var/lib/lxc/android $ tree . . ├── artifacts │ └── init ├── config ├── post-stop.sh ├── pre-start.sh ├── rootfs └── run</code></pre> Check if the container is getting detected by Android using lxc-info android</code>. If it is getting detected, we should be ready to go. Finally, setup the lxc.android service so that the container auto-starts on boot. cd /etc/init.d ln -s lxc lxc.android rc-update add lxc.android boot</code></pre> As a bonus, we can also setup SSH for accessing Gentoo when Android boots. The dialhome script mentioned below can be found here</a>. umask 077 mkdir -p ~/.ssh mount /dev/block/by-name/userdata /var/lib/android/data/ cat /var/lib/android/data/ssh/id_ed25519.pub >> ~/.ssh/authorized_keys cp dialhome /var/lib/android/data/ssh/ chmod u+x /var/lib/android/data/ssh/dialhome chown 2000:2000 /var/lib/android/data/ssh/dialhome umount /var/lib/android/data rc-update add sshd default</code></pre> We are done with Gentoo now, exit it and unmount its pseudofs exit ## Now back in Android's Shell umount -f /data/gnu/proc umount -l /data/gnu/sys umount -l /data/gnu/dev</code></pre>Final Steps</h2> We are almost done now. Reboot to your Recovery and flash the boot-mod.img</code> to the boot</code> partition. Fastboot users could also flash it via</li> </ol> fastboot flash boot boot-mod.img</code></pre> While you're in the Recovery mode, fire up adb again and patch the init.rc</code> present in /system</code> according to the patch here</a>. Keep in mind that we do not need to patch Android's fstab</code> if you're following this setup.</li> </ol> And then finally cross your fingers and reboot. If Android boots, congrats, you just joined the SharkBait party. But it gets even better. Final Thoughts and Scopes of Improvement</h2> If Android does not boot, check the logs at /data/gnu/var/logs</code>. Check dmesg</code> for kernel logs, rc.log</code>, messages</code> for Gentoo's logs, lxc/hellalxc.log</code> and lxc/android.log</code> for LXC logs. Currently SharkBait is at a very young stage. Encryption and SELinux are not supported right now. But this is subject to change once SharkBait matures. Also if this entire process seems a bit lengthy, I should inform that when the bootstrap-init and sar-preinit merges into the official SharkBait workflow, this install will take significantly lower efforts and would be a breeze to the porters as well as the users of SharkBait. Lastly I should give credits to the official guide</a> on which this install guide is based on. Please have a look at it as well as my other blogs to understand how this works under-the-hood. Obligatory Neofetch </figcaption> </figure> </div>

Other things</h3>
Home manager has a rich set of options to explore. I am currently using also aerc and alacritty through it.</p>

Future</h2>
Home manager is nice and unobtrusive piece of software which is fine for setting up on foreign hosts. Let's move ahead and integrate home-manager with `nix-darwin</code> to also make system level changes.</p>`

Pitfalls</h2>

UART on Xiaomi Redmi Note 7

Android Boot High Jinks: And What it Means for SharkBait

Installing Gentoo on Android

Gunwant Jain - @wantguns - blog

Installing NixOS Remotely with nixos-anywhere

Single flake, multiple hosts

System-Level macOS Configuration with nix-darwin

Getting Started with Nix Home Manager on macOS

Setting up my homelab and cloud with Nix

Building LMDB JNI for Jetson Nano

Building ONNX Runtime for Jetson Nano

Building</h2>
I hate to use docker for automating VM purposes, but I acknowledge it as an ad-hoc method for this instance.</p>
We will be compiling natively for arm64 in a qemu image which is running inside a docker container.</p>

Pitfalls</h2>

IPC between Termux and Other Android Apps using ZMQ

Final Report - GSoC 2020

Work Covered</h2>

Building Android: BTS

Starting System-as-Root Android in a LXC container

Booting Gentoo from SAR-Preinit; or Debugging the init, the Hard Way

Gunwant Jain - @wantguns - blog

Installing NixOS Remotely with nixos-anywhere

nixos-facter</h3> I am going to generate the hardware configuration for mintaka</code> using nixos-facter:</p>

Single flake, multiple hosts

modules/features/default.nix</h3> I'll declare all the abstractions for home manager options in this file.</p>

modules/features/implementation.nix</h3> I'll implement the declared options in this file.</p>

System-Level macOS Configuration with nix-darwin

Getting Started with Nix Home Manager on macOS

Other things</h3> Home manager has a rich set of options to explore. I am currently using also aerc and alacritty through it.</p>

Future</h2> Home manager is nice and unobtrusive piece of software which is fine for setting up on foreign hosts. Let's move ahead and integrate home-manager with nix-darwin</code> to also make system level changes.</p>

Setting up my homelab and cloud with Nix

Concepts</h3> I have had multiple scattered thoughts on what designs fits best for my needs, and I have realised:</p>

Dev logs</h3> I am going to start blogging about my journey, here is timeline i'll keep updating:</p>

Building LMDB JNI for Jetson Nano

Building</h2> Once again, I hate to use docker for automating VM purposes, but I acknowledge it as an ad-hoc method for this instance.</p> We will be compiling natively for arm64 in a qemu image which is running inside a docker container.</p>

Pitfalls</h2>

Building ONNX Runtime for Jetson Nano

Building</h2> I hate to use docker for automating VM purposes, but I acknowledge it as an ad-hoc method for this instance.</p> We will be compiling natively for arm64 in a qemu image which is running inside a docker container.</p>

Pitfalls</h2>

IPC between Termux and Other Android Apps using ZMQ

Final Report - GSoC 2020

Work Covered</h2>

Building Android: BTS

Starting System-as-Root Android in a LXC container

Prerequisites</h2> This setup expects some prerequisites which are as follows:</p>

Preparing the boot image</h2> Clone the SAR-Preinit repository</p>

Preparing Gentoo for Android</h2> Chroot into Gentoo and start setting it up.</p>

Install LXC</h3> At the time of writing this blog, LXC's version 4 does not work for the setup. I would recommend to install the version 3.</p>

Final Steps</h2> We are almost done now.</p>

Booting Gentoo from SAR-Preinit; or Debugging the init, the Hard Way

UART on Xiaomi Redmi Note 7

The Need of a Serial Console</h2> My first attempt at porting SharkBait for SAR failed. I was confident about the approach but had trouble finding out what went wrong. If only somehow I could get the kernel logs before my phone went into the bootloop, I could quickly provide fixes.</p>

PL2303HXA</h2> After that, I connected the jumper wires to the PL2303HXA convertor I bought and quickly fired up a screen</code> session, excited to look at the console output.</p>

Android Boot High Jinks: And What it Means for SharkBait

This poses some challenges</h3> What do we use now as lxc-container's rootfs</li>

Installing Gentoo on Android

Other things</h3>
Home manager has a rich set of options to explore. I am currently using also aerc and alacritty through it.</p>

Future</h2>
Home manager is nice and unobtrusive piece of software which is fine for setting up on foreign hosts. Let's move ahead and integrate home-manager with `nix-darwin</code> to also make system level changes.</p>`

Building</h2>
I hate to use docker for automating VM purposes, but I acknowledge it as an ad-hoc method for this instance.</p>
We will be compiling natively for arm64 in a qemu image which is running inside a docker container.</p>