• [deleted]

  • Post #1
[h]
================== S6/66 - ZFS - KDE/PLASMA ON GENTOO LINUX ==================
[/h]

[h]
-------------------
PART1: ZFS
-------------------
--------------------------------------------------------------------------------------------------
Method 1: Partitioning Scheme: MBR / ZFS: Non-mirrored device
--------------------------------------------------------------------------------------------------
[/h]

From a live CD with ZFS support
live ~# fdisk -l
live ~# cfdisk /dev/sda
dos
new
1G
primary
type
be Solaris boot
new
primary
type
bf Solaris
write
yes
quit

WARNING: Device ID values here are only displayed for demonstration
WARNING: Replace BYID_SDA1 and BYID_SDA2 values with your own value
live ~# ls -l /dev/disk/by-id

lrwxrwxrwx 1 root root  9 Sep 27 11:38 ata-VBOX_HARDDISK_VBfda948a7-fd4176fe -> ../../sda
lrwxrwxrwx 1 root root 10 Sep 27 11:38 ata-VBOX_HARDDISK_VBfda948a7-fd4176fe-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 Sep 27 11:38 ata-VBOX_HARDDISK_VBfda948a7-fd4176fe-part2 -> ../../sda2

live ~# export BYID_SDA1=ata-VBOX_HARDDISK_VBfda948a7-fd4176fe-part1
live ~# export BYID_SDA2=ata-VBOX_HARDDISK_VBfda948a7-fd4176fe-part2

WARNING: Option autotrim=on is used for SSD
WARNING: Remove -o autotrim=on for HDD
live ~# zpool create -f -o cachefile=/tmp/zpool.cache -o ashift=12 -o autotrim=on -d -o feature@ allocation_classes=enabled -o feature@ async_destroy=enabled -o feature@ bookmarks=enabled -o feature@ embedded_data=enabled -o feature@ empty_bpobj=enabled -o feature@ enabled_txg=enabled -o feature@ extensible_dataset=enabled -o feature@ filesystem_limits=enabled -o feature@ hole_birth=enabled -o feature@ large_blocks=enabled -o feature@ lz4_compress=enabled -o feature@ project_quota=enabled -o feature@ resilver_defer=enabled -o feature@ spacemap_histogram=enabled -o feature@ spacemap_v2=enabled -o feature@ userobj_accounting=enabled -o feature@ zpool_checkpoint=enabled -O acltype=posixacl -O atime=off -O canmount=off -O compression=lz4 -O devices=off -O encryption=off -O mountpoint=none -O normalization=formD -O relatime=on -O xattr=sa bpool /dev/disk/by-id/${BYID_SDA1}

live ~# zpool create -f -o cachefile=/tmp/zpool.cache -o ashift=12 -o autotrim=on -O acltype=posixacl -O atime=off -O canmount=off -O compression=lz4 -O devices=off -O mountpoint=none -O normalization=formD -O relatime=on -O xattr=sa rpool /dev/disk/by-id/${BYID_SDA2}
.
live ~# zfs list

NAME USED AVAIL REFER MOUNTPOINT
bpool 360K 832M 96K none
rpool 312K 47.0G 96K none
.
live ~# zfs create -o canmount=off bpool/BOOT
live ~# zfs create -o mountpoint=/boot -o canmount=noauto bpool/BOOT/gentoo
live ~# zfs create -o canmount=off rpool/ROOT
live ~# zfs create -o mountpoint=/ -o canmount=noauto rpool/ROOT/gentoo

live ~# zpool set bootfs=rpool/ROOT/gentoo rpool
.
live ~# zfs list

NAME USED AVAIL REFER MOUNTPOINT
bpool 648K 831M 96K none
bpool/BOOT 192K 831M 96K none
bpool/BOOT/gentoo 96K 831M 96K /boot
rpool 648K 47.0G 96K none
rpool/ROOT 192K 47.0G 96K none
rpool/ROOT/gentoo 96K 47.0G 96K / 
live ~# zpool get bootfs rpool

NAME PROPERTY VALUE SOURCE
rpool bootfs rpool/ROOT/gentoo local
.
live ~# zpool export -a

live ~# zpool import -f -d /dev/disk/by-id -R /mnt rpool
live ~# zpool import -f -d /dev/disk/by-id -R /mnt bpool
.
live ~# zfs list

NAME USED AVAIL REFER MOUNTPOINT
bpool 648K 831M 96K none
bpool/BOOT 192K 831M 96K none
bpool/BOOT/gentoo 96K 831M 96K /mnt/boot
rpool 732K 47.0G 96K none
rpool/ROOT 192K 47.0G 96K none
rpool/ROOT/gentoo 96K 47.0G 96K /mnt
.
live ~# zfs set canmount=on rpool/ROOT/gentoo
live ~# zfs set canmount=on bpool/BOOT/gentoo
.
live ~# zfs get canmount rpool/ROOT/gentoo

NAME PROPERTY VALUE SOURCE
rpool/ROOT/gentoo canmount on local 
live ~# zfs get canmount bpool/BOOT/gentoo

NAME PROPERTY VALUE SOURCE
bpool/BOOT/gentoo canmount on local
.
live ~# zfs mount rpool/ROOT/gentoo
live ~# zfs mount bpool/BOOT/gentoo
.
live ~# zfs get mounted rpool/ROOT/gentoo

NAME PROPERTY VALUE SOURCE
rpool/ROOT/gentoo mounted yes - 
live ~# zfs get mounted bpool/BOOT/gentoo

NAME PROPERTY VALUE SOURCE
bpool/BOOT/gentoo mounted yes -

[h]
-------------------------------------------------------------------------------------------
PART2: Gentoo stage3 with S6-66 installation/configuration
-------------------------------------------------------------------------------------------
[/h]

WARNING: Stage3 version archive here is only displayed for demonstration
WARNING: Download the last version from https://www.gentoo.org/downloads/
live ~# wget -c https://bouncer.gentoo.org/fetch/root/all/releases/amd64/autobuilds/20200923T214503Z/stage3-amd64-20200923T214503Z.tar.xz -P /tmp
live ~# wget -c https://bouncer.gentoo.org/fetch/root/all/releases/amd64/autobuilds/20200923T214503Z/stage3-amd64-20200923T214503Z.tar.xz.DIGESTS -P /tmp
live ~# wget -c https://bouncer.gentoo.org/fetch/root/all/releases/amd64/autobuilds/20200923T214503Z/stage3-amd64-20200923T214503Z.tar.xz.DIGESTS.asc -P /tmp
live ~# wget -c https://bouncer.gentoo.org/fetch/root/all/releases/amd64/autobuilds/20200923T214503Z/stage3-amd64-20200923T214503Z.tar.xz.CONTENTS.gz -P /tmp
.
live ~# gpg --verify /tmp/stage3-amd64-20200923T214503Z.tar.xz.DIGESTS.asc
gpg: keybox '/root/.gnupg/pubring.kbx' created
gpg: Signature made Wed 23 Sep 2020 11:01:05 PM GMT
gpg: using RSA key 534E4209AB49EEE1C19D96162C44695DB9F6043D
gpg: Can't check signature: No public key 
live ~# gpg --keyserver hkps://keys.gentoo.org --recv-keys 534E4209AB49EEE1C19D96162C44695DB9F6043D

live ~# gpg --verify /tmp/stage3-amd64-20200923T214503Z.tar.xz.DIGESTS.asc
gpg: Signature made Wed 23 Sep 2020 11:01:05 PM GMT
gpg: using RSA key 534E4209AB49EEE1C19D96162C44695DB9F6043D
gpg: Good signature from "Gentoo Linux Release Engineering (Automated Weekly Release Key) <releng@ gentoo.org>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 13EB BDBE DE7A 1277 5DFD B1BA BB57 2E0E 2D18 2910
Subkey fingerprint: 534E 4209 AB49 EEE1 C19D 9616 2C44 695D B9F6 043D
.
live ~# cd /tmp

live ~# sha512sum -c stage3-amd64-20200923T214503Z.tar.xz.DIGESTS
stage3-amd64-20200923T214503Z.tar.xz: OK
stage3-amd64-20200923T214503Z.tar.xz: FAILED
stage3-amd64-20200923T214503Z.tar.xz.CONTENTS.gz: OK
stage3-amd64-20200923T214503Z.tar.xz.CONTENTS.gz: FAILED
INFO: Those failed are from whirlpool


OPTIONAL: whirlpool > Install rhash
live ~# wget -c https://bouncer.gentoo.org/fetch/root/all/releases/amd64/autobuilds/20200923T214503Z/stage3-amd64-20200923T214503Z.tar.xz.DIGESTS -P /tmp

live ~# whirlpool-hash -a -c stage3-amd64-20200923T214503Z.tar.xz.DIGESTS
--( Verifying stage3-amd64-20200923T214503Z.tar.xz.DIGESTS )--------------------
stage3-amd64-20200923T214503Z.tar.xz OK
stage3-amd64-20200923T214503Z.tar.xz OK
stage3-amd64-20200923T214503Z.tar.xz.CONTENTS.gz OK
stage3-amd64-20200923T214503Z.tar.xz.CONTENTS.gz OK
--------------------------------------------------------------------------------
Everything OK
.
live ~# cd /mnt

live ~# tar xpvf /tmp/stage3-amd64-20200923T214503Z.tar.xz --xattrs-include='*.*' --numeric-owner

live ~# mount -t proc none proc
live ~# mount --rbind /sys sys
live ~# mount --rbind /dev dev
live ~# env -i HOME=/root TERM=$TERM $(which chroot) . bash -l
.

CHROOT
live /# export PS1="(chroot) $PS1"

(chroot) ~# nano /etc/resolv.conf
nameserver 80.67.169.12
nameserver 80.67.169.40
.
(chroot) ~# emerge --sync
.
OPTIONAL: if enough RAM (minimum:8G - recommended:12G+) 
(chroot) ~# nano /etc/fstab
tmpfs /var/tmp/portage tmpfs rw,nosuid,noatime,nodev,size=8G,mode=775,uid=portage,gid=portage,x-mount.mkdir=775 0 0 
(chroot) ~# mount tmpfs
.
(chroot) ~# emerge -DNuav @ world
(chroot) ~# emerge --depclean

(chroot) ~# emerge -av gentoo-kernel-bin
.
(chroot) ~# emerge -av zfs
(chroot) ~# ls /lib/modules/5.4.64/extra/zfs/
zfs.ko
.
(chroot) ~# nano /etc/portage/make.conf
USE="libzfs" 
(chroot) ~# emerge -av sys-boot/grub:2
.
(chroot) ~# nano /etc/dracut.conf.d/default.conf
hostonly="yes"
stdloglvl=4 
(chroot) ~# nano /etc/dracut.conf.d/zfs.conf
nofsck="yes"
add_dracutmodules+="zfs"
omit_dracutmodules+="btrfs"
.
(chroot) ~# eselect kernel list
[1] linux-5.4.64 *
.
(chroot) ~# dracut --force --kver 5.4.64
(chroot) ~# lsinitrd /boot/initramfs-5.4.64.img |grep zfs
zfs
-rw-r--r-- 1 root root 324 Sep 27 15:59 etc/udev/rules.d/90-zfs.rules
-rwxr-xr-x 1 root root 4856 Sep 27 16:00 lib/dracut-zfs-lib.sh
-rwxr-xr-x 1 root root 265 Sep 27 16:00 lib/dracut/hooks/cleanup/99-zfs-needshutdown.sh
-rwxr-xr-x 1 root root 2077 Sep 27 16:00 lib/dracut/hooks/cmdline/95-parse-zfs.sh
-rwxr-xr-x 1 root root 2549 Sep 27 16:00 lib/dracut/hooks/mount/98-mount-zfs.sh
-rwxr-xr-x 1 root root 2615 Sep 27 16:00 lib/dracut/hooks/pre-mount/90-zfs-load-key.sh
-rwxr-xr-x 1 root root 480 Sep 27 16:00 lib/dracut/hooks/shutdown/20-export-zfs.sh
drwxr-xr-x 2 root root 0 Sep 27 16:10 lib/modules/5.4.64/extra/zfs
-rw-r--r-- 1 root root 4773304 Sep 27 15:54 lib/modules/5.4.64/extra/zfs/zfs.ko
lrwxrwxrwx 1 root root 15 Sep 27 16:10 lib64/libzfs.so.2 -> libzfs.so.2.0.0
-rwxr-xr-x 1 root root 491432 Sep 27 16:00 lib64/libzfs.so.2.0.0
lrwxrwxrwx 1 root root 20 Sep 27 16:10 lib64/libzfs_core.so.1 -> libzfs_core.so.1.0.0
-rwxr-xr-x 1 root root 26696 Sep 27 16:00 lib64/libzfs_core.so.1.0.0
-rwxr-xr-x 1 root root 26880 Sep 27 16:00 sbin/mount.zfs
-rwxr-xr-x 1 root root 136952 Sep 27 16:00 sbin/zfs
.
(chroot) ~# mkdir /etc/modules-load.d
(chroot) ~# nano /etc/modules-load.d/zfs.conf
zfs
.
(chroot) ~# emerge -av dev-vcs/git

(chroot) ~# emerge -av eselect-repository
.
(chroot) ~# mkdir /etc/portage/repos.conf
(chroot) ~# nano /etc/portage/repos.conf/66tools-overlay.conf
[66tools-overlay]
location = /var/db/repos/66tools-overlay
sync-type = git
sync-uri = https://framagit.org/architekt/66tools-overlay.git
.
(chroot) ~# eselect repository list
[4] 66tools-overlay @ 
(chroot) ~# eselect repository enable 66tools-overlay
(chroot) ~# emerge --sync 66tools-overlay
.
(chroot) ~# cd /etc/portage
(chroot) ~# mkdir package.accept_keywords
(chroot) ~# cd package.accept_keywords
(chroot) ~# ln -s /var/db/repos/66tools-overlay/profiles/package.accept_keywords/s6-66tools.accept s6-66tools.accept
.
(chroot) ~# eselect profile list
(chroot) ~# eselect profile set --force 35
(chroot) ~# eselect profile list
[35] 66tools-overlay:default/linux/amd64/17.1/s6-66 (exp) *
.
(chroot) ~# emerge -DNuav @ world
(chroot) ~# emerge --depclean
.
(chroot) ~# 66-tree -n boot
(chroot) ~# 66-enable -t boot -C -F boot@ system

WARNING: Replace HOSTNAME and TZ with your own value 
(chroot) ~# EDITOR=nano 66-env -e -t boot boot@ system
HOSTNAME=gentoo66
TZ=Europe/Warsaw
FSTAB=!yes
ZFS=!yes
ZFS_IMPORT=!scan 
(chroot) ~# 66-enable -t boot -F boot@ system
.
(chroot) ~# emerge -av dhcpcd-66serv

(chroot) ~# 66-tree -nEc root
(chroot) ~# 66-enable dhcpcd
.
(chroot) ~# 66-intree
Name : boot
Initialized : no
Enabled : no
Starts after : None
Current : no
Allowed : root
Symlinks : svc->backup db->backup
Contents : tty@ tty12 system-hostname mount-run populate-run mount-tmp populate-tmp mount-proc mount-sys
populate-sys mount-dev mount-pts mount-shm populate-dev mount-cgroups 00 system-hwclock
modules-kernel system-random modules-system system-sysctl udevd-log udevd udevadm
system-fontnkey devices-zfs-import-scan devices-zfs system-Devices all-Mount system-fsck
mount-fstab all-System mount-rw mount-netfs local-loop local-sethostname local-time
local-authfiles local-tmpfiles local-dmesg all-Local all-Runtime All tty-rc@ tty2 tty-rc@ tty1
boot@ system

Name : root
Initialized : no
Enabled : yes
Starts after : None
Current : yes
Allowed : root
Symlinks : svc->backup db->source
Contents : dhcpcd-log dhcpcd
.
(chroot) ~# nano /etc/66/init.conf
VERBOSITY=4
.
(chroot) ~# zpool set cachefile=/etc/zfs/zpool.cache rpool
(chroot) ~# zpool set cachefile=/etc/zfs/zpool.cache bpool
(chroot) ~# zdb -C -U /etc/zfs/zpool.cache

WARNING: Device ID value here is only displayed for demonstration 
(chroot) ~# ls -l /dev/disk/by-id
ata-VBOX_HARDDISK_VB07bea66a-b3ec443d -> ../../sda 
(chroot) ~# grub-install /dev/disk/by-id/ata-VBOX_HARDDISK_VB07bea66a-b3ec443d
(chroot) ~# grub-mkconfig -o /boot/grub/grub.cfg

(chroot) ~# grub-probe /
zfs
.
(chroot) ~# passwd

(chroot) ~# exit
END OF CHROOT

.
live ~# pwd
/mnt
.
live ~# zgenhostid
live ~# cp /etc/hostid /mnt/etc/
.
live ~# umount -lR /mnt/proc
live ~# umount -lR /mnt/dev 
live ~# umount -lR /mnt/sys
.
live ~# cd /               
live ~# zpool export bpool
live ~# zpool export rpool
.
live ~# poweroff
.

Reboot and enjoy Gentoo with S6-66 and ZFS :)

[h]
----------------------------------------------------------------------------------------------
PART3: KDE/Plasma with 66 services installation/configuration
----------------------------------------------------------------------------------------------
[/h]

WARNING: Choice of USE flags will not be discussed here.

WARNING: Rust can be extremely long to build depending on your cpu. 
gentoo66 ~# emerge -av1 rust-bin

gentoo66 ~# emerge -av xorg-server
WARNING: Useful for a quick test before installing Plasma and KDE components 
gentoo66 ~# emerge -av jwm

gentoo66 ~# emerge -av zsh

gentoo66 ~# useradd -m -G wheel,video -s /bin/zsh <username>

gentoo66 ~# passwd <username>

gentoo66 ~# emerge -av boot-user-66serv

gentoo66 ~# 66-tree -nEc <username>-session

gentoo66 ~# 66-enable -t <username>-session -F boot-user@ <username>

gentoo66 ~# 66-env -t <username>-session -e nano boot-user@ <username>
# DISPLAY_MANAGER=sddm
CONSOLE_TRACKER=elogind
XDG_RUNTIME=!no
DESKTOP_CMDLINE=!jwm 
gentoo66 ~# 66-enable -t <username>-session -F boot-user@ <username>

gentoo66 ~# reboot
WARNING: Now login to your <username> session
gentoo66 ~# startx

  • [deleted]

  • Post #2
66 is fully functional and all services are available on Gentoo.
I use it daily on one of my machines so I personally take care of updating the repository which is available here:
https://framagit.org/architekt/66tools-overlay
a month later

  • [deleted]

  • Post #3
Available ebuild update
# emerge --sync 66tools-overlay
Does encryption work on zfs the same as with an ext4 volume, but for the entire disk I assume?
The only change in the setup is to alter the command to encryption=on

I used to swear by ext4 but lost some stuff recently by rm -rf a partition that had another still mounted on, so I am ready for some new learning. I am split between redcore and clover to be honest, I am not about ready to be compiling kernels and browsers.

  • [deleted]

  • Post #5
Hi fungal_net

I do not personally use encryption so I really don't know how encryption works on ext4. Are you talking about LUKS ?
I also don't use native ZFS encryption so I could not really speak on this subject.

https://zfsonlinux.org/manpages/0.8.5/man8/zfs.8.html
https://zfsonlinux.org/manpages/0.8.5/man8/zfs.8.html# lbAO

Some things to know. Settable properties with some exception inherit their value from the parent dataset. Properties must be carefully chosen at creation time because some of them cannot be altered later or under certain conditions. Changing some property only affects newly-written data.

https://www.medo64.com/2020/11/testing-native-zfs-encryption-speed-20-10/
I used to swear by ext4 but lost some stuff recently by rm -rf a partition that had another still mounted on
ZFS snapshot and you will be able to tease rm -rf :D

Although even if Linux OpenZFS is gradually taking a different road than the Oracle ZFS, it's already a good and well written documentation to consider.

https://docs.oracle.com/search/?q=managing+zfs
I was thinking about LUKS but didn't know of the enryption it provides on its own, so I may look through it.

Did zfs start with sun and then oracle picked it up, or did they buy sun on its dawn? Some of the literature about zfs points to sun (sunos-solaris) related info.

  • [deleted]

  • Post #7
Did zfs start with sun and then oracle picked it up, or did they buy sun on its dawn?
ZFS (old:Zettabyte file system) combines a file system with a volume manager. It began as part of the Sun Microsystems Solaris operating system in 2001. Large parts of Solaris – including ZFS – were published under an open source license as OpenSolaris for around 5 years from 2005, before being placed under a closed source license when Oracle Corporation acquired Sun in 2009/2010.

src wikipedia
I remember there was a solaris fork that started under free license before Oracle forked it as non-free ... but the name is no longer solaris.,,, I think it is openindiana I am thinking of. As I see on DW it had zfs since day 1. But would this be openzfs as was opensolaris ... or they just incorporated the oracle development?
10-15 years earlier who would have thought that names such as sun or sgi, or cray, would vanish from computing when they appeared superior enough to swallow hp dell ibm and microsoft .... let alone the flakiest of all ever ... Apple!

What the hell happened in the past 20y and where was I?
a month later
This OpenZFS 2.0 has incorporated zstd compression functionality within the file system's management/tools.
I am searching whether there is anyone being critical of this development. and don't seem to find any.
It is a bit crazy how this crapy system is thown out of facebook as open/free sw and it is everywhere within a year.

https://www.bsdnow.tv/366 This is a podcast that brags about this "new feature".
21 days later
gentoo's recipe for s6 is wrong, everything up to skalibs builds and s6 fails because on the configure there is a library difference: /usr/lib/skarnet/sysdeps should be /usr/lib64/skarnet/sysdeps ?

  • [deleted]

  • Post #11
Hi fungal_net

It depends on the architecture you building the software. This is managed by the variable $(get_libdir)
For a 64-bits architecture, it's /usr/lib64/ and for a 32-bits it's /usr/lib/
I am not that experienced in emerge, and I have left everything as default (except for removing elogind) and this is the only such error I've gotten. I followed your instructions and enabled your architect and slalibs execline s6 ... come in as dependencies to oblibs 66 .... and it got stuck. I tried to do it manually with make install and although it seems to build right it will not install, I guess this is blocked and must be done through emerge and I don't know how.

  • [deleted]

  • Post #13
Well, that strange. Few days ago I have built all the s6-66 stack from my repository for my ARM board and everything worked as expected.
Could you pastebin the output of 
emerge --info
and the s6 build log ( when building fails you get a path to the log /var/tmp/portage/sys-apps/s6/temp/build.log )

Thx
https://termbin.com/k26m

Those are 2 logs one is after I tried to change the configure file

By mistake I overwrote the emerge --info
I reached my limit in consolework, since to get rid of elogind I even had to remove xorg-server and I haven't mastered the art of modifying ebuilds to build without the "pests"

  • [deleted]

  • Post #15
You are not building s6 from my repository as shown by the log:
Package: sys-apps/s6-2.9.2.0
Repository: gentoo
Could you please pastebin the output of: 
eselect profile list
3 years later
eric unstickied the discussion .

Powered by Obarun