Not saying that you fell for clickbait but this is going to be a quite opinionated piece that didn’t go anywhere next to the scrutiny a LWN article would have gone and merely represents a summary of my findings on this particular topic so far.
Let’s go back to the beginning of the 2000s: Printing under Linux/Unix was a mess but a new savior on the horizon, CUPS (The “Common Unix Printing System”), developed by Michael Sweet of Easy Software Products, saw the light of day to change this. CUPS was going to make things more convenient and also bring with it new features such as browsing that would eventually turn out to be especially useful on print servers. “Print what”? Let’s briefly detour…
A print server, a print server and a print server go into a bar…
Look kids, back then in the stone ages affordable printers (and I will assume you know what a printer is, otherwise you wouldn’t be reading this) outside the Enterprise segment didn’t have network connectivity built-in and you had to augment them with devices like this one:
This was called a “print server”, which, for some printers such as the legendary HP LaserJet 4/5/6 (yes, there was a time when HP wasn’t crap), could often also be retrofitted later in special slots. Prices fell and nowadays most printers except the cheapest ones have this functionality integrated. But what functionality exactly? If my printer has a LAN port, I can just print to it directly, no?
Yes, you can. Because whatever client you print from is likely to implement spooling of jobs to the printer. Spooling is required because, unless you discover some larger storage device such as a HDD or SSD in your printer, it requires documents to be pre-rendered to the printer’s native language and served piece-by-piece. Spooling is also required since you may want to print multiple documents as once and these print jobs need to be processed one after another to avoid them getting mixed up.
But also: no, you can not. At least not if you’re not the only one using that printer. The printer is likely to simply block and ignore data from a second client when it’s currently busy processing the first client’s job. So it makes sense to have only one client at any time actually spooling data to the printer. It also makes sense because you then have a central place for accounting purposes, any additional processing that might be deemed necessary such as watermarking printouts and most of all so you have one place to cancel jobs instead of potentially dozens. That client is called… a “print server”. Eh? This too? Don’t @ me, it wasn’t my idea.
And it gets even funnier: your Linux or Windows client itself is a “print server”, too, since there’s virtually no difference functionality-wise, it’s just that we pick one dedicated machine to be the “central” print server. So we have your client (a “print server”) spooling to a dedicated machine (also a “print server”) spooling to a printer’s network interface (also a “print server”). Confused yet? From here on, If I say “print server”, I mean the software running on the dedicated machine, which, of course, will be CUPS.
“Ok, I see that enterprises may benefit from this all, but why would I need this at home in 2022?” Because even today it’s beneficial if a.) clients don’t have to install specific printer drivers but can defer to the print server to do the actual rendering for the specific printer, and, even better, b.) clients don’t have to be manually setup for the print server and its printers but can discover them automatically.
Printer discovery the legacy way
Assuming adequate firewall configuration on servers and clients, in old CUPS versions service announcements, or “browsing” as CUPS would misleadingly call them, would simply be configured with the BrowsingOn and BrowseAddress directives. A print server configured this way periodically broadcasted its existence and the printers it offered, and the CUPS client installed by default with most Linux desktop installations would automatically pick them up and create local CUPS queues for them with a suitable name, i.e. including the print server’s hostname along with the printer name. I will not go as far as claiming that this was totally suitable for each and every enterprise but it worked just fine for those I got to know. It also was inherently compatible with road warriors changing environments since besides DHCP supplying IP and name server information the different sites’ printers would appear and disappear automagically.
One point to note is that the announcement/discovery mechanism was specific to CUPS and platforms running CUPS. Windows clients, for example, were unable to take part. While a CUPS Postscript driver was released and Windows eventually learned IPP itself at some point, its built-in discovery mechanisms still were restricted to NetBIOS and SMB1 and thus required Samba on the server side. This German news piece on the CUPS 1.1.6 release suggests that there were plans to follow up with a “full IPP client” implementing CUPS’ discovery later, but to my knowledge that was never actually implemented and alas would still have been 3rd party software that needed installation.
An Apple a day keeps manual configuration away
Narrator: Apple Inc. enters the stage.
Apple didn’t just integrate CUPS into their “new” Unix-based MacOS, in 2007 they bought the rights to the software and hired the team behind it, which led not only to the natural end of Easy Software Products but also to fears as laid down in this LWN article:
“The real threat, perhaps, is that Mr. Sweet will find himself carrying a lot of Apple-specific responsibilities […]”
I can not concur with or reject this prediction but what we can attest is that in spite of even offering server hardware themselves at some time, and influenced by their experiences with AppleTalk and the migration to TCP/IP, CUPS-wise their focus seems to have been on a different target group, environments that were to benefit from the envisioned results of the IETF Zeroconf working group (which existed from 1999 to 2004):
“Environments where administration is impractical or impossible, such as in the home or small office, embedded systems ‘plugged
together’ as in an automobile, or to allow impromptu networks as between the devices of strangers on a train.”
Although the working group was dissolved without reaching agreement, three technologies later emerged from its work:
- Dynamic Configuration of IPv4 Link-Local Addresses (RFC 3927), also known as Automatic Private IP Addressing
- Multicast DNS/mDNS (RFC 6762)
- DNS-based Service Discovery/DNS-SD (RFC 6763, dns-sd.org)
The first of these technologies was intended to enable communication between involved parties in the first place by automatically negotiating IP addresses from the 169.254.x.x range without the need for a third party coordinating their assignment. Except that in the end everyone did have such a third party in form of the DHCP server in their Internet router, often in the form of dnsmasq. Because everyone got Internet. So for IPv4, link-local addresses played less of a role, by contrast to IPv6.
Having enabled communication amongst each other, clients next potentially want to use each others’ services, for which they need to have a common vocabulary describing these services. This is where DNS-SD jumps in by leveraging the existing well-established Domain Name System (DNS) and its record types, specifically PTR, SRV and TXT records, to construct an analogy to a directory and file structure: a root “directory” containing “subdirectories” for each offered service, e.g. “ipp”, and “files” in these subdirectories for each particular implementation of a service, e.g. a particular printer/print server speaking “ipp”. It is worth noting here that we explicitly talk about services, not hosts, since that’s what users usually care about. Consequently, service names are not limited by the restrictions that hostnames have to obey, e.g. service names can contain spaces.
In the context of the ZeroConf stack, DNS-SD works particularly well with the third technology, mDNS. mDNS is similar to DNS but instead of sending unicast requests to a DNS server and receiving unicast replies, it uses multicasting, packets sent to a multicast group (224.0.0.251 in the case of IPv4) that clients need to explicitly listen to if they’re interested. Everyone can send to the multicast group and thus publish, everyone can listen and thus discover, i.e. there is no central server anymore.
Apple implemented mDNS and the other standards under the Bonjour banner (formerly called Rendezvous), for Linux a well-known student of computer sciences called Lennart Pöttering developed Avahi long before he and others invented systemd. Microsoft, by contrast, while implementing APIPA early on didn’t bother about mDNS until Windows 10 or so and instead came up with their own, supposedly “more advanced” technology called Link-Local Multicast Name Resolution which in fact was not only less advanced but also not exactly secure. And even with Windows 10, mDNS support is still developing. There was a Bonjour version for Windows limited to discovering printers but that had to be downloaded and installed separately, which noone did.
Think global, act .local
So how do these three work together on your print server then? CUPS announces its printers, e.g. a HP LaserJet 6N (we’re old-school here, after all) as hplj6n._ipp._tcp.local. Your CUPS client picks that up and resolves the name via nss-mdns to a pair of SRV and TXT records, which as described above, are part of the DNS-SD spec. The SRV record contains the hostname and port of the print server to connect to, the TXT record follows Apple’s Bonjour printing specification and contains e.g. the name of the printer queue on the print server. And if it didn’t catch previous multicasts, the client can also actively discover active services by sending a mDNS request for _services._dns-sd._udp.local which resolves to the PTR record for the “root directory”, from where it can go to the “ipp directory” and enumerate printers itself.
There is however one culprit with mDNS and DNS-SD: while clients no longer need to know at least one IP address of a DNS server, there still has to be an agreement on a common domain name to be used by both clients and servers, both to distinguish the mDNS “namespace” from unicast DNS hostnames and to have a common entry point. Don’t forget that in a ZeroConf world, clients have set up IP addresses (and netmasks), but they don’t have the concept of a common domain name, i.e. the suffix to append to the _services._dns-sd._udp. part, so the mDNS specs had to jump in and specify a default here. Which became .local – much to the joy of numerous Windows admins who had followed Microsoft that at one time used contoso.local as example domain in their Active Directory documentation and now would have to change that for conformity, a painful process.
I’ve got 99 problems but my TLD is one
Now I explained a lot about Apple buying CUPS and pushing forward on ZeroConf technologies but I didn’t name the elephant in the room yet: this new mDNS/DNS-SD technology was declared the new kid on the block for service discovery and the legacy, CUPS-specific method deprecated for a number of reasons: resolving problems in DNS-less networks and broadcasts slowing down Wifi networks and generally not being welcomed by administrators.
So where’s the problem then?
The problem is that now it’s not just the DNS-less networks that get a new .local namespace but also existing, “managed” networks with unicast DNS. If your entire enterprise DNS resolves around a well laid-out topology, e.g. .sfo.americas.company.com, having a new second namespace .local to monitor for problems is not exactly groovy.
Also, any serious enterprise IT is about providing a controlled environment, for support but especially also for security reasons. A .local namespace in which anybody can publish and claim any name sounds anything but secure to me. You can protect official servers in your unicast DNS zone with DNSSEC all the way but if I can spoof printsrv01.local just by being faster in registering that name, because of short but interruptive network issues or because I simply turned off the real machine for a while, and finance department clients start sending their confidential reports to my rogue laptop because even though there is TLS-protected IPP, there is no mutual authentication, that really does not sound secure to me.
And honestly, isn’t .local just ugly? Even IT staff can have aesthetic considerations…
But DNS-SD is also specified for Unicast?
Yes, it is. You can specify DNS-SD records for your unicast DNS-served TLD, as described e.g. by Dan Jackson for printer discovery. But there are two obstacles to consider here:
- The whole ad-hoc approach behind ZeroConf sort of implies that DNS-SD advertised services appear when they are actually accessible and disappear when not. For example, if I remove a CUPS printer queue, that change should be communicated right away so that clients remove their matching local queue and users can no longer submit jobs to it.
With unicast DNS, this not only implies that the branch used for DNS-SD, e.g. .sfo.americas.company.com, supports RFC 2136-style DNS updates as performed with nsupdate. Which in turn requires not only a fully-featured DNS server (dnsmasq won’t do) but also TSIG-based authentication as specified in RFC 2845-based to be set up.
It also means that, given the one-way communication pattern of DNS, clients have to periodically poll for updates since the DNS server can’t notify them. And DNS records for stale services would have to be deleted manually since using the DNS records’ TTL fields for this purpose results in either excessive stale records, due to the TTL being too high, or excessive refreshes by clients, due to a too low TTL.
Both issues have been addressed by two proposed DNS extensions, DNS long-lived queries and DNS Update Leases. Although both never made it to the status of a RFC, they have been implemented in form of an auxiliary daemon to the bind name server called dnsextd which apparently shipped with Mac OS X and as part of mDNSResponder, but attempts to get this running under Linux were not really fruitful and to my knowledge none of the well-known DNS servers implemented them.
You could of course forego the dynamic nature and simply set up static DNS-SD records, accepting the disadvantages of users never knowing whether a printer is actually available. Maybe your printer queues don’t change that much and you could even script the DNS-SD record setup from within your Ansible role that sets up the print server.
- But the bigger problem is that you now need this information to arrive at the client. Avahi is mDNS only, no?
Nope! Avahi supports unicast DNS as well, calling it “wide area” mode like the dns-sd.org guys. But the restriction from back then when Lennart implemented it still applies: Avahi supports wide area discovery but not publishing, so software such as CUPS can not use Avahi to publish its printers and you’d probably have to hack up something that communicates with CUPS and calls nsupdate on its behalf.
But that’s not all, of course. On all clients Avahi now needs to know the new domain to use for DNS-SD instead of .local. The obvious thing to do was described by the dns-sd.org guys:
Fortunately, in most cases, the necessary configuration is already present. When clients get a response packet from the local network’s DHCP server, there’s a “domain” parameter in that packet, and it’s typically used as the client’s default DNS search domain. DNS-SD clients also use it to discover their default DNS service discovery domain. So, in most cases, all the required client configuration is already in place. What remains is simply to add a few service discovery records to the authoritative name server(s) for the domain given in the client’s DHCP configuration.
Bonjour implemented that, so Avahi will, too, right?
Nope! It doesn’t. Yawn. You will have to explicitly configure it in /etc/avahi/avahi-daemon.conf, either in the domain-name directive, thereby disabling .local and as a consequence mDNS browsing at all, or in browse-domains, although that didn’t seem to work in my testing. On every client, for CUPS’ sake. Sure, you could argue that in enterprise environments, clients are under sysadmin control and deployed with custom settings anyway, but think of SOHO environments with visiting guests – if guests have to reconfigure their Avahi domains in order to receive the site’s DNS-SD records, they could just as well configure the CUPS server to poll or even the printer queues directly. Not exactly satisfying.
CUPS vs. cups-filters
There is one more development that I haven’t addressed yet: a lot of functionality was removed from CUPS itself because Apple saw no need for it and had to go into a similarly-named but separate project, cups-filters. This includes not only legacy CUPS printer announcing and discovery but also modern mDNS-based discovery using Avahi and querying LDAP servers for printer information: they all are now implemented by a separate daemon called cups-browsed. cupsd itself only handles talking to Avahi for mDNS-based service announcements anymore.
What’s interesting in this context is the one of deprecation: is legacy CUPS announcing/discovery deprecated by Apple/CUPS only or also the cups-filters project? The documentation seems to hint at the latter, especially since the OpenPrinting.org project behind cups-filters seems to be headed towards totally different issues anyway.
Conclusion
Unless I have overseen something essential, I can’t help but conclude that this all translates to CUPS’ printer discovery having become pretty much unusable for enterprise Linux environments. For the moment one can certainly still enable legacy CUPS browsing which may have its deficits but is about a 100 times easier to setup, but the long-term perspective looks bad unless Avahi, after a long time, still gets the missing features, wide area service publishing and inferring search domains from FQHNs.
Although we may have some time left since the departure of the CUPS lead developer, Mr. Sweet, from Apple at end of 2019 caused press to be under the impression that CUPS development has seemingly dried up…