A thought about todays documentation practices based on an example, but really, a matter of common practice.

Yesterday I finally got to upgrade my system after a few days of having doubts about completing my usual daily update procedure. I have a disk with personal data that's using ZFS as a file system and as we all know, the ZFS license is not compatible with Linux, so there's people porting the Solaris layer every day for us to be able to use it.
For Arch there's a specific port called archzfs so you don't need to compile from AUR which is nice but often lacks behind the mainstream kernel for it always has to be compiled specifically for one kernel at a time. For people like me who don't want to wait a few days, there's the automated testing repo which keeps track just nice.
Now the ZFS on Linux package had a version update with a pretty changing behavior in its latest version: from before to this version it would fuse two packages together. (Specifically spl-utils got merged into the ZFS package)
This had me doubt several days if I should update or not because pacman prompted me whether to replace x with y (which is quite common in package updates and usually safe to do) but then also notified me that package y would now be in conflict with package x and if I wanted to delete x. Wait what?

So this is where my philosophical journey begins. I eventually found out on the archzfs Github Repo─where somebody else opened an issue to make sure that that's right before going yes and yes─that it'd be fine to go ahead and delete package x. At that moment I felt that I wanted to comment on how I would have liked to see this severe change in behavior documented on either the Arch Wiki or the archzfs Github Repo.
The only one who answered right away was the author of the very same issue with a rather harsh tonality about how "It's on their front page [He referred to the ZOL Github Tags, Ed.] and reddit is very excited about it." And whether that was "[...] good enough?"
I did not want to argue there, so we eventually got to share some comments about documentation options in pacman. One of the main package maintainers also commented in a constructive manner later on. But no, I do not think it is "good enough".
We got stuck on how it was not possible in pacman to really reach the user before installing the package. But I don't think that's the point. There's a good old README right in the top directory of almost every Github project where you could make such announcements and delete them once they are out of date.

There are a lot of very conventional places for text. Whenever I do a little script, a program or some publication I usually try to ask myself: "What is the possibly most stupid thing one could do with this?" and then try to come up with some verbose information on how to prevent that. As a matter of fact I therefore know that it can be incredibly annoying to keep track of documentation whenever doing updates even more so small ones. But sometimes even the small ones can confuse your users.

I feel that it has become common practice for us developers and maintainers to expect the user to be investigative. For whenever he gets confused about my program he should have informed himself better, or searched the internet for several hours if there really was no fanatic already writing about that matter before coming to me, or having looked upstream; when in reality years old buzz talk is all about user experience. I for one do visual design, typography, front-end development with preprocessors, Node, CMS, templates, linux system services and a bunch of other crap. Am I lazy? Should I have searched the internet well enough when not knowing whether my system will break after accepting to delete a conflicting package?

In the early automobile days and also the early computing era user and service manuals were thick heavy printed books with every possible maintenance step and technical detail documented. Today you won't even find a service manual for your car and it is business practice to prevent you from fixing anything anywhere, be it your car your phone your book shelf or even your very house. We are digital though, we are open source and the written word, like in this forum or any other website, is supposed to be common good and shared intelligence. In fact it is pretty much probable that somebody already wrote about your product. I do not think more and bigger is just better though. I'm a designer at heart. It is about quality. In this case it's about prominence and priority, about user experience.
User experience is providing the user with solutions as fast as possible in a straight forward way as possible with minimal effort. If I cared about my users I could not expect that there may be other sources writing and documenting about my product.

What do you think? Am I overreacting? Is documentation getting better or worse in your point of view? As users, should we put more effort in searching for documentation? As creators, should we put more effort in providing prominent documentation at the source?

This is a general thought and not meant to become a rant about specific cases so please share your thoughts in a general manner.
> This is a general thought and not meant to become a rant about specific cases so please share your thoughts in a general manner.

My usual thoughts are crude, sometimes rude, harsh, confrontational, ..... I'll have to call my ex- to give me some more characterizations :)

Since you didn't ask for a specific case, here is one. Gparted, an old friend, was just shifted from a 0,3... version to 1.0. Together with it this time came a huge pkg called gtk3mm and not knowing what this may be and is so important, and huge, I downloaded its documentation. Beautifully written, a joy to browse through it. I bet the Gnome team includes a whole writing documentation department where various scientists, linguists, technical writers, are employed. It is all about quality, and effectiveness, and man-hours, and man-dollar-days. An infinitesimal budget luxury in IBM's empire.
Ok, seriously, I thought about this some more, because I am used to the sneaky way Marian has in slipping in "simple" puzzles for complicated issues.

I went beyond documentation, and its tremendous variation. You see in ms-win-nt (last time I had taken a serious look at MS) you look and read at one piece of documentation and you know what to expect and how all are organized. In **ux/ix you have various sources and styles. People can't even agree whether they all use --help or -h or how man xxxx-software is structured. You read one and say wow, you read another and you are finished with more questions than before you opened it up. The majority of users try to use software without even taking a quick look at documentation, especially before they try it. If you are like me you try it, it fails, you look at options trying to guess the right syntax, it fails again, and then you end up reading a man page just searching for what you want to do not what in totality the software can do. Then some months go by and you think you know it all and it is getting 1-2-3-8 upgrades and you never look again but both the software and docs. have changed. Pacman now is not what pacman was a few years back.

I think it is safe to conclude that quality documents need quality readers. But also quality software can only be respected by very demanding and knowledgeable users. So you have s6 and you have systemd, and by the vast majority systemd is better. I remember when I thought it should be so easy to create an .iso of a system as using dd (like: # dd if=/dev/sdb3 of=~/myarch.iso). Then I realized it wasn't. I was on debian back then and there was a tool (which surprisingly can not be found in Arch) called xorriso (not like in Spanish sausage). The guys that make these tools are geniuses in file system peculiarities and experts in iso-9660. What has stuck in my mind is how complex the subject is and how good the documentation is for those who want to learn the art (https://www.gnu.org/software/xorriso/man_1_xorriso_devel.html). It was good reading, as opposed to really confusing and boring reading, and pretty effective if you got into it.

So one of the problems we face in linux is that the documentation is inconsistent. There is a difference reading subjects from 3 different volumes of an encyclopedia, and 3 different books on the same subject. Here we have 3 different authors of 3 different subjects. Sometimes inconsistency is just as bad as bad quality. On top of learning something new you have to learn how to read about something new in a new way. I hope we agree. Do you like wikipedia? I think it is one of the largest achievements of humanity since the Gutenberg invention. With a $50 machine you can install obarun and a browser, and a few dvd's of the latest wikipedia and allow people to learn more than they can in a lifetime. I think it is one TB for the English version which is the largest, followed by Spanish (maybe old data, and Chinese has grown).

I mentioned above a hint of the constraints for all this and the fallacy of comparing industrial mass produced tools (of profitability) against one-off pieces of craftsmanship. A Pininfarina prototype with a 67' Chevy Chevelle. The Chevelle is both better documented and much more reliable. I'll take the Pinin-farina with all of its faults and lack of documentation. But back to Gnome documentation and format. Have you thought how really sloppy code, with tons of bugs and possible situations of them causing an error or a crash, can be covered up with a gui? It may evolve into 60% of the code becoming a patch to cover up and deal with bugs and errors. The gui installer (for example) may pop-up a fancy colorful and decorated window saying that "not all software could be installed from the repository at the time, would you like to go back and restart the installation or would you like to continue any way", while the true error messages may be two pages and it is all because the installer doesn't work too well under certain conditions. The documentation may be so fancy that it strikes you as the world's friendliest linux installer. Search around for Calamares, some people will not even try a distribution unless it comes with Calamares. I think it is a turd and when it runs into a fault it doesn't give you a clue of how bad it screwed up. But it sells, meaning "people" like it. It has colors and buttons, and drop menus, and "options". The first time I installed Obarun and got it right, I thought it was so much fun I did it a second time with different options. I couldn't believe that it could be so simple and so direct with as much freedom to choose (I never installed compton ever since). I admit I don't like the latest change of those yes/no half screens as I did the older style. I like it when it runs into an error it exits and tells you exactly why it did and at what point it did. Ok, if you ask me to make an installation really quick now and make sure it boots the first time I would just use pacman, but if you ask me to install slackware I would probably rely on the installer.

Since you compared auto manuals I remember this sad woman, school teacher, whose Citroen was sick, took it to official service and got a diagnosis that was about as much or more as her weekly salary, for a whole new distributor. I heard the car huffing and puffing, I raised the hood, I could almost hear sparks ticking around the cap, and there it was, a hairline crack of the cap. The new cap I put on, about $10, made it run perfect, I threw in a set of cables as they looked fried because she couldn't afford them. She was miserable for days driving around at 40kph burning all this fuel.
The guys at the official service were no crooks, they were young mechanics, following what the school taught them, listened to their duties prescribed by the supervisor and did what they were told. They plugged in the computer at the ECU and the code they got was for a faulty distributor. They weren't lying. If a bug had crawled in and burned up a contact it would still show the same. The instructions at the professional documentation of the official service centers in this 21st century do not have any instructions in rebuilding distributors, only replacing the whole!

Ok, I stop here, I already said too much, it is hard for me to discuss things and not make them a political issue.

I HATE IT when well groomed developers, distro master-designers, deal with a problem a user is having and they recommend to do a reinstallation. There is nothing in linux/unix that can't be fixed. Proposing a re-installation may be just good when you don't have the time for a proper diagnosis and repair. Devs shouldn't make those decisions for the user, who may want to spend the time and learn, but may need a little direction on where to start looking. MS tells people to reinstall because you can't diagnose shit that is all locked up and hidden.

I think the art of diagnosis is similar, car, computer system. bicycle, human. Just machines!
Thank you for this very honest comment. I was about to ask you for something more general :P
I wouldn't mind it to be longer even. I can totally agree with every sentence, I've had similar experiences and found similar things. I can not quite read if you are rather satisfied or left wishing when it comes to documentation in our weird software niche which is Linux but I agree that inconsistency is a huge point and can be frustrating. On the other hand it's so hard to try to be consistent as a creator. Github long ago introduced guidelines for README's. As a designer I was tought to create style guides for future consistency. But as you say we are many, and we write different each with different backgrounds.
Indeed Wikipedia is something of its own and I think it's incredible, but careful, it's also a propaganda and a lobby platform. Critics on specific programs, technologies, enterprises and similar are swiftly moderated by people closely related to the subject and even converted into the possibly least harmful comment or even positivated (do I explain myself?).
As a designer I think GUIs are good. And good GUIs are even better. But as a good designer I also want the engineers to do their job well. I've had my problems with Calamares before. Deepin has a massive visual manual installed with the system, which first threw me off because of the file size of the install,...but it's really rather nice for beginners.
I would never say there's no documentation around, in fact generally speaking there is so many written word present overall on the internet, documenting even the most absurd little details of even unknown practices, that as you say, it's more than a lifetime of knowledge. It's a heavy topic as we tend to be lazy beings (after all that's what we make tools and programs for...). You for one are used to write a lot of documentation for example but I tend to write more code. That simple fact makes we want to avoid explaining on how something works and rather make it as simple as possible. But even the simplest things in life can become complex rather quick and that's when the user has no control anymore.
Nice read, thank you Fungal. Now for the others, where are you?! :D
While waiting for others to read 3 page comments, ....
marianarlt wroteCritics on specific programs, technologies, enterprises and similar are swiftly moderated by people closely related to the subject
Imagine on subjects like history, economics, society, anthropology, how much of a tool of bias and propaganda can there be, if on innocent things like natural science, technology, there is an interest for bias and propaganda. Still though you may find specialty cases where there was enough uproar and reaction to a bias that the higher moderators to accommodate and diffuse a situation that was ridiculing wikipedia, they would split a topic into two, or more, definitions and perceptions. This was slightly more democratic than say "encyclopedia Britanica" where an unknown selection of authors would be the ultimate authority on a subject. In wikipedia you might see one authority telling the other they are full of shit, and this is public, and it can continuously be fought upon and change. It is a step on the right direction. 50-60 years ago you would be called a fool or insane for contesting what a major encyclopedia was defining. Kids in school would be taught how light traveled by books and teachers who were never exposed to modern physics. If you said anything about photons you may have gotten a bad grade.

Back to docs. One thing I think we are both missing here is audience. When Debian incorporates xorriso documentation into its system, or arch with something like zfs peculiarities, they have a different audience in mind than Ubuntu writing a document about a gui that incorporates xorriso functionality or manipulation of data structure on a zfs volume. I think the difference between a compiler manual and gparted manual is such that reflects on the difference of audience. Same with a car manual, the service-shop manual for a BMW m3 and the one Hanes sells as a service manual appeal to different audience. I used to have boxes of Hanes manuals for cars and motorcycles that had no real shop manuals, not in the US anyway. I am talking about Renault Caravelle and Bultaco Alpina manuals :)
I remember how some same sections existed in all manuals that a professional mechanic would never read, like how to seat valves on an iron head or adjust dwell on points.

Obarun and Artix are relieved with the major burden of having to document things for entry-level users, clearly stating the distributions are aimed at seasoned linux users/admins. If you have a gap in knowledge, in a subject in Arch, and you have no clue where to even find the information in arch-wiki, it might help to find some sense in Manjaro documents. The two appeal to different audience. How do you define such grouping and where is the line drawn to separate the two groups I don't know. I think Eric may have an idea. Should time and effort be wasted from other Obarun needs to write a document on how to use commands such as ls, cp, mv, rmdir, ln, grep, etc...? There is one relatively new Obarun user who a little while ago was asking in Artix to be referred to a good place to learn linux. One should learn how a library is organized before they start walking up and down the isles looking for books. Does it help having a really good zfs or ext3 manual when there is no reference on what a file system is? Someone may have used linux for 10 years and try for the first time FreeBSD, or Solaris. A list comes up with which file-system to use and none look familiar. How do you decide while the installer is running?
Well you mentioned it so I'll just throw in my point of view. I personally think it's a mistake (as in marketing) to state that any certain software should be "aimed at experts". It is something that threw me way off when I started Linux. So I avoided things like Arch, Slackware, FreeBSD, Gentoo etc. When in reality they are the best documented pieces of operating systems out there. I mean every crap program that maybe only a handful of people would use has a step by step guide often with troubleshooting hints and bug findings. This is where my actual topic kicks in. I did not want to get at Obarun at all. I think there are more important steps for now, but it'd be great to have a consistent emphasis on important stuff. This is also part of the web design which I proposed and did not yet start to work on.
As you say there's absolutely no point in explaining what great options there are to basic Linux programs (fzf is actually quite powerful btw...) and also as Eric once said: Obarun documentation should really focus on where there are differences to Arch only.
Back to topic: Ain't the Arch wiki like basically the best source for overall operating software documentation?! That's my whole point, there's so much to this. I mean it has articles on cross platform software useful to major OS users just as much as general Linux users. Even when not using Arch, when I had to read up about software to get a lvl up on my learning curve (lol) I'd very often end up reading the Arch wiki. Maybe it's as you say and we're actually moving towards a collaborative common knowledge base (Borg anyone? hahaha!) It's intriguing to me.
marianarlt wrote (Borg anyone? hahaha!)
As they say in Mexico, there is a difference between a Borg from below and a Borg from above ;)
Really i like to read you guys, not about your point of view or the subject discussed but how i can learn english :D.

Maybe i can bring an another point of view to the subject. I totally agree about the documentation lack on this world. Not even only on computer science but on every part in the life and mostly on personal evolvement. The knowledge is the most important part on this world, the only real resources renewable , the only one which multiply itself when you share it, the only one which allow to be free. Give a fish to an homeless man and he will eat one time, show him how to fish and he will eat every day!
I talk about real information, not all this disinformation that you can find everywhere and let people in the ignorance. Let knowledge, more control...

well back to the docs focus. I remember the first time that i tried a Linux system. It was Ubuntu, coming from windows like the majority of people, the distro is easy to test. After one months of try, i decided to install linux on my main station. Well, the first question was: "Which distro i pick?". The answer was Arch. Never heard about a console, shell scripts, a path,... nothing. I will always remember when i read the Arch installation wiki this command : mkdir /mnt/{home,boot}. It took me a while to understand what's mean {home,boot} and in particularity the role of '{}' bracket :D. Finding this information was horrible and that the problem.
Where the wiki should start?Knowledge assumptions is necessary and this fact is a pity because it let some people on the back. The other side is that you can not explain every single command,line,case from the beginning.
So, an important factor enter in the scene: the voluntary of the readers and this is not controllable and only depends of him. You can make any great wiki which explain all the things from the base, you always find something missing for a man. Also, the contrary is right. A man with important knowledge will waste time to find an information into a ton a wiki page.
Well, now an another trouble come: where to find the information? A ton and ton and ton of same page explaining the same thing with the same manner with the same syntax and so on. For example you want to search how to make redirection of file descriptor in C. Taking google, you will found 90% of the time the same answer: "ho man this is simple , use this library". Oh, ok but i want to understand how it works and make it in low-level. Suddenly the relative page of the subject is decreasing quickly :D.
Wikipedia turn around this problem and centralize the information. This is useful but allow the enter of the devil more easily too. I mean, a wrong information will be propagandized quickly. The best example is the TV....(sorry i go back to the subject)

And now, documentation from a user point of view and documentation from a dev point of view diverges importantly. The user is the reader, the dev is the writer and this two position cannot be merged correctly. Example, 3 lines of code written in 5 minutes can take 1 hour to be documented correctly because this 3 lines of code interact with 1000 lines of code and you can multiply this time by the coefficient of the user knowledge(back the the first trouble: where the wiki should start).
Futhermore, what happens when the projects is make by a Team where each member take care of only one part of the project? Each member can write the doc about his part but not the whole thing, and so an another trouble come: a man with sufficient knowledge and with a general view of the projects need to write the doc(i simplify the thing).
Ok, good, we have this guy, but a good dev doesn't mean a good professor...and at this point the docs begin to sucks...
Also the "best" person to write the doc is the dev itself but if you ask at a dev to write a documentation, 90% of them(including myself,so 90%,00001) will answer: "i have no time for that, i will make it later" and at this point the projects begin to sucks.... because without good documentation you don't give at the project the opportunity to be knew.(SO MANY THANKS GUYS FOR YOUR HELP ABOUT THAT :D).

I would like to talk about the support of the doc and in particularity manpages. "What's the hell going wrong with this?" answer:"The hell is too old now, you need a new one". Seriously manpages is horrible to read, horrible to find something into it and worse literally boring to write with an horrible syntax to learn. But when you don't provide manpage people are angry and ask again and again and again to it. This fact bring the last(i'm optimist) problem: Accepting the changes! and, on this world, it's very difficulty to make because change come with fear, fear of the new, fear of the fact to force to go ahead of the limits.
What do not use html(i pick this one but it can be another format)? every system now have a good html reader even in console and it come with link,video,picture,easy to write even with a few tag knowledge... but the changes....

Well, i stop here because i have work to do on 66... the documentation of the new incoming tools:"I HAVE NO TIME FOR THIS" haaaaaaaaaaaaa
cheers
eric wroteReally i like to read you guys, not about your point of view or the subject discussed but how i can learn english :D.
chicken looking confused
Hahahahahahaha
eric wrotemkdir /mnt/{home,boot}. It took me a while to understand what's mean {home,boot} and in particularity the role of '{}' bracket :D. Finding this information was horrible and that the problem.
Careful here I'd say. For me as a coder it's easy to write and understand something like:
[[ ! $a ]] && printf "%s %s%d\n" $b || $a=$b
when even an advanced user may not yet have come across, or understand logical operators and have difficulty reading something like this. It may be a bad example you think because almost nobody will need such constructions if it wasn't for development but:
mkdir /mnt/{home,boot}
is really only an abbreviation as most here know. And a lot of commands are really rather verbose as they date back to when GUIs were not yet a thing. "make directory /mnt/home" and then "make directory /mnt/boot" Man now THAT's verbose!
Actually I think GUIs have parted us a bit from this verboseness. There's a lot of commands and programs that are so un-intuitive to use because of their grammar that they include a lot of learning process. But then again you cannot just make a "better" from your point of view or "advanced" mkdir with the same command, because well, it's already there.
So things like cfdisk (really great!) or fd (a sped up find) or similar are born.
When presenting such commands to the broad public I think we should refrain from using advanced trickery like bracket expansion, logical operators and the like. If a specific user is sufficiently advanced he can do that anyways.
(Now you might discuss about how on the contrary it could be educational, but as both of you mentioned, that's not your/our job)

Also your comment about MAN ("manual" talk about verbose!) I can partially understand and partially disagree with :D I do appreciate an extensive MAN page although I totally agree that it's a very outdated format for creators and users alike. Difficult to change such a massive implementation of documentation like MAN. Thanks for all your thoughts guys, keep 'em coming :D
There is no any problem with docs as any info is a *doc* already, to some extent - from a line of code to man and in between. All is needed is a channel of communication and a possibility to ask and receive an answer. And a doc / man may be upgraded as any pkg in arch.

Powered by Obarun