warning for latex-provided ts1?

[kberry]

I was looking at ltnews39 and was reminded of the need for font package maintainers to provide TS1 subset declarations. If that's started happening, fine. But if not (as I suspect), I wondered about having LaTeX emit a warning ("tell the font maintainer to update their package") when its subset declaration is used.

Just an idea. If this is already being done, sorry. I couldn't easily check right now.

[FrankMittelbach]

To be honest I think it would perhaps better if we do collect the email addresses of the dozen or less people that produce font packages and write to them privately and ask them to please make that update (which is not that difficult I would say, given that there is the code to generate the right setting for you). Putting such warnings it each an every log file means a) a lot of noise for all users and b) it is actually not that easy to determine when it would be appropriate: the default may as well be the correct value c) I doubt that much is reaching the maintainers this way.

It essentially just requires getting the right email addresses (or perhaps the github accounts and assign them here) ... once that happened I'm happy write that email.

Any takers for the collection part? If not I may do it at some point myself, but right now I don't have the bandwidth.

[mbertucci47]

Had a moment so figured I'd write down those I could think of.

• Carl-Clemens Ebinger (https://github.com/ccebinger)
• Clea F. Rees (https://github.com/cfr42)
• Bob Tennent (rdt at cs.queensu.ca)
• Antonis Tsolomitis (atsol at aegean.gr)
• Michael Sharpe (msharpe at ucsd.edu)

[kberry]

A few others come to mind (I feel weird writing out email addresses for gh to harvest, but I guess all of them are publicly known already): Boguslaw Jackowski (jacko at bop.com.pl) https://ctan.org/author/jackowski Arash Esbati (arash at gnu.org) https://ctan.org/author/esbati Mohamed El Morabity (melmorabity at fedoraproject.org) https://ctan.org/author/morabity Hirwen Harendal (harendalh at hotmail.com) https://ctan.org/author/harendal There are many more (dozens or hundreds, I would guess). I could auto-extract the info from the Catalogue, but I suspect the people mentioned already would cover a good portion of what's in use nowadays. And once they start doing it, hopefully the idea will become more widely known. So maybe it is better to start with those than worry about a longer list up front. I will also add a note to the TL package contribution page, and ask CTAN to do the same on theirs, although I don't expect that to have any particular effect. Last paragraph in: https://tug.org/texlive/pkgcontrib.html#complex kb

[cfr42]

Some package authors do provide declarations, but they typically seem to be in .sty files.

• Clea F. Rees (https://github.com/cfr42)

Packages I'm responsible have declarations in the various .sty files¹. For various reasons, I'm not going to put them into .fds unless somebody convinces me the advantages outweigh the disadvantages. For the record, I discussed this with two members of the LaTeX Project, but I'm not currently persuaded.

Hirwen Harendal ... https://ctan.org/author/harendal

Hirwen Harendal is not responsible for the LaTeX support, but for the fonts. This is explained in the packages' documentation, so that name shouldn't be on your list.

gentium-tug doesn't include declarations, but the default is recommended, so they would only be necessary if there's some reason to include them even in that case².

To be honest I think it would perhaps better if we do collect the email addresses of the dozen or less people that produce font packages and write to them privately and ask them to please make that update (which is not that difficult I would say, given that there is the code to generate the right setting for you).

Note that tools which auto-generate .fd files don't typically support adding the declarations, so it is not necessarily the best place for them from a maintenance perspective. Almost all the .fd files I make are auto-generated by fontinst. (berenisadf is the exception.) Experience suggests that anything which requires me to remember to change an auto-generated file prior to upload is likely to result in erroneous uploads³.

There is also the question of what to then do with the result the test gives you, which is not so obvious, imho.

_{¹Very few (probably none) are 'correct' according to the test. This is one reason I don't want them in the .fd files. Another is that there's no way to say 'use whatever declaration FM uses for font X for font Y'. Admittedly, there's probably not great demand for such a feature.
²Testing gentium-tug was how I discovered the typo in the test file.
³This isn't my reason for not putting them in the .fd files as fontinst.lua can already add variable scaling factors, which fontinst doesn't support either.}

[car222222]

Many thanks, Clea, for all this e tra information: very useful.

[cfr42]

grep -l DeclareEncodingSubset tex/latex/*/*.sty tex/latex/*/*.fd > /tmp/ts1-decs

on a very recently updated TeX Live produces:

tex/latex/algolrevived/algolrevived.sty
tex/latex/baskervaldadf/baskervald.sty
tex/latex/berenisadf/berenis.sty
tex/latex/cfr-lm/cfr-lm.sty
tex/latex/chivo/Chivo.sty
tex/latex/cochineal/cochineal.sty
tex/latex/electrumadf/electrum.sty
tex/latex/erewhon/erewhon.sty
tex/latex/gfsdidot/gfsdidot.sty
tex/latex/libris/libris.sty
tex/latex/newpx/newpxtext.sty
tex/latex/newtx/newtxtext.sty
tex/latex/newtxtt/newtxtt.sty
tex/latex/romandeadf/romande.sty
tex/latex/scholax/scholax.sty
tex/latex/tudscr/tudscrfonts.sty
tex/latex/venturisadf/venturis.sty
tex/latex/venturisadf/venturis2.sty
tex/latex/venturisadf/venturisold.sty
tex/latex/xcharter/XCharter.sty
tex/latex/ysabeau/ysabeau.sty
tex/latex/zlmtt/zlmtt.sty

[I removed results for base.]

[cfr42]

Note that I initially put them in the .sty files because I copied what I found in 2e's sources. Just as I put scaling factors in the .fd files because I copied from psnfss (as it says in my source code, I think).

So I don't think it should be surprising that authors who've done this have used .sty files rather than .fds. To avoid that, I think you'd at least need comments in base .sty files and sources2e clearly recommending people not to put the lines in .stys. If I'm updating a font package, I'm much more likely to be reading your source code than I am ltnews.

[FrankMittelbach]

Packages I'm responsible have declarations in the various .sty files1. For various reasons, I'm not going to put them into .fds unless somebody convinces me the advantages outweigh the disadvantages. For the record, I discussed this with two members of the LaTeX Project, but I'm not currently persuaded.

well, the main reason is that it is technically the place where it belongs, meaning in many cases you can use fonts without an additional .sty support file, e.g., by just doing \fontfamily{family}\selectfont in which case LaTeX would load the necessary .fd files but nothing else.

Note that tools which auto-generate .fd files don't typically support adding the declarations, so it is not necessarily the best place for them from a maintenance perspective. Almost all the .fd files I make are auto-generated by fontinst. (berenisadf is the exception.) Experience suggests that anything which requires me to remember to change an auto-generated file prior to upload is likely to result in erroneous uploads.

Are you using the original fontinst of the tool my Marc?

In either case, I agree with you that it would be best to have that automated as part of the autogeneration of the .fd files.
But that could, I would think, done by passing the generated fd through a script that appends the right declaration to the file at the end.

There is also the question of what to then do with the result the test gives you, which is not so obvious, imho.

I'm sure, the script and documentation could be improved, but what exactly do you find difficult or non-obvious from, say, the output

TS1 encoding subset for Cochineal-TLF (ok)
Use sub-encoding 5\relax 

It could spit out the full declaration line, which is probably better, but otherwise?

[FrankMittelbach]

Note that I initially put them in the .sty files because I copied what I found in 2e's sources. Just as I put scaling factors in the .fd files because I copied from psnfss (as it says in my source code, I think).

So I don't think it should be surprising that authors who've done this have used .sty files rather than .fds. To avoid that, I think you'd at least need comments in base .sty files and sources2e clearly recommending people not to put the lines in .stys. If I'm updating a font package, I'm much more likely to be reading your source code than I am ltnews.

It is always a bit difficult to figure out where people are going to look and how much you therefore should duplicate in the documentation.

fntguide.pdf that contains the full description of what should go into an .fd file say

Maintainers of font bundles that include TS1 encoded font files should add an appropriate declaration 
into the corresponding ts1family.fd file, because otherwise the default subencoding is assumed, which is 
probably disabling too many glyphs that are actually available in the font.

but yes perhaps it could be extended and explicitly say that this is the right place and not a .sty file, because the font could be used without that .sty file.

I don't think the 2e sources use that declaration anywhere in a sty file, if we do it is a mistake.

[davidcarlisle]

I don't think the 2e sources use that declaration anywhere in a sty file, if we do it is a mistake.

the only place we use it is in textcomp.sty (it's not in the ts1 fd files in base) in fact in texlive tex/latex area I only see it in packages

$ grep -l 'DeclareEncodingSubset{TS1}' */*{sty,fd}
algolrevived/algolrevived.sty
base/latexrelease.sty
base/textcomp-2018-08-11.sty
base/textcomp.sty
baskervaldadf/baskervald.sty
berenisadf/berenis.sty
cfr-lm/cfr-lm.sty
chivo/Chivo.sty
cochineal/cochineal.sty
electrumadf/electrum.sty
erewhon/erewhon.sty
gfsdidot/gfsdidot.sty
libris/libris.sty
newpx/newpxtext.sty
newtx/newtxtext.sty
newtxtt/newtxtt.sty
romandeadf/romande.sty
scholax/scholax.sty
tudscr/tudscrfonts.sty
venturisadf/venturis.sty
venturisadf/venturis2.sty
venturisadf/venturisold.sty
xcharter/XCharter.sty
ysabeau/ysabeau.sty
zlmtt/zlmtt.sty

so while I agree with the argument that it's better to be in the fd files to allow direct use of the font, the current situation is rather different

[FrankMittelbach]

the only place we use it is in textcomp.sty (it's not in the ts1 fd files in base)

in textcomp it is setting overall defaults according to the package option not explicit settings for individual families and that is just a compatibility thing because that is what the package did in its early days.

There are no declarations in ts1 fds in base because technically they don't have to (as the kernel has them and they are therefore alway present).

However, I agree for consistency and to make a good example we should probably change that.

[...]
so while I agree with the argument that it's better to be in the fd files to allow direct use of the font, the current situation is rather different

well, yes, most fonts have no settings and a few have them now in their support packages. But the main reason to write ltnews39 and also this issue here is to change that situation.

[cfr42]

Packages I'm responsible have declarations in the various .sty files1. For various reasons, I'm not going to put them into .fds unless somebody convinces me the advantages outweigh the disadvantages. For the record, I discussed this with two members of the LaTeX Project, but I'm not currently persuaded.

well, the main reason is that it is technically the place where it belongs, meaning in many cases you can use fonts without an additional .sty support file, e.g., by just doing \fontfamily{family}\selectfont in which case LaTeX would load the necessary .fd files but nothing else.

Yes, I know that's the reason. (And in general I agree that's preferable.)

Are you using the original fontinst of the tool my Marc?

Sorry. I don't understand this question.

In either case, I agree with you that it would be best to have that automated as part of the autogeneration of the .fd files. But that could, I would think, done by passing the generated fd through a script that appends the right declaration to the file at the end.

Yes, it could. If I was going to do it, I'd add it to fontinst.lua, since the script already edits the .fd files to add variable scaling factors for ADF fornts.

I'm sure, the script and documentation could be improved, but what exactly do you find difficult or non-obvious from, say, the output

No, nothing. I didn't mean the output wasn't clear.

The reason I don't want to put them in the .fd files is that the declarations are much easier to override if they are in the .sty files.

[cfr42]

I don't think the 2e sources use that declaration anywhere in a sty file, if we do it is a mistake.

the only place we use it is in textcomp.sty (it's not in the ts1 fd files in base) in fact in texlive tex/latex area I only see it in packages

latexrelease.sty:

\DeclareEncodingSubset{TS1}{?}{9}
\DeclareEncodingSubset{TS1}{ccr}     {0}
\DeclareEncodingSubset{TS1}{cmbr}    {0}
\DeclareEncodingSubset{TS1}{cmr}     {0}
\DeclareEncodingSubset{TS1}{cmss}    {0}
\DeclareEncodingSubset{TS1}{cmtl}    {0}
\DeclareEncodingSubset{TS1}{cmtt}    {0}
\DeclareEncodingSubset{TS1}{cmvtt}   {0}
\DeclareEncodingSubset{TS1}{pxr}     {0}
\DeclareEncodingSubset{TS1}{pxss}    {0}
\DeclareEncodingSubset{TS1}{pxtt}    {0}
\DeclareEncodingSubset{TS1}{qag}     {0}
\DeclareEncodingSubset{TS1}{qbk}     {0}
\DeclareEncodingSubset{TS1}{qcr}     {0}
\DeclareEncodingSubset{TS1}{qcs}     {0}
\DeclareEncodingSubset{TS1}{qhvc}    {0}
\DeclareEncodingSubset{TS1}{qhv}     {0}
\DeclareEncodingSubset{TS1}{qpl}     {0}
\DeclareEncodingSubset{TS1}{qtm}     {0}
\DeclareEncodingSubset{TS1}{qzc}     {0}
\DeclareEncodingSubset{TS1}{txr}     {0}
\DeclareEncodingSubset{TS1}{txss}    {0}
\DeclareEncodingSubset{TS1}{txtt}    {0}
\DeclareEncodingSubset{TS1}{lmr}     {1}
\DeclareEncodingSubset{TS1}{lmdh}    {1}
\DeclareEncodingSubset{TS1}{lmss}    {1}
\DeclareEncodingSubset{TS1}{lmssq}   {1}
\DeclareEncodingSubset{TS1}{lmvtt}   {1}
\DeclareEncodingSubset{TS1}{lmtt}    {1} % missing TM, SM and pertenthousand
\DeclareEncodingSubset{TS1}{ptmx}    {2}
\DeclareEncodingSubset{TS1}{ptmj}    {2}
\DeclareEncodingSubset{TS1}{ul8}     {2}
\DeclareEncodingSubset{TS1}{bch} {5} % tofu for blank, ohm
\DeclareEncodingSubset{TS1}{futj}{5} % tofu for blank, interrobang/down, ohm
\DeclareEncodingSubset{TS1}{futs}{5} % tofu for blank, ohm
\DeclareEncodingSubset{TS1}{futx}{5} % probably (currently broken distrib)
\DeclareEncodingSubset{TS1}{pag} {5} % tofu for blank, interrobang/down, ohm
\DeclareEncodingSubset{TS1}{pbk} {5} % tofu for blank, interrobang/down, ohm
\DeclareEncodingSubset{TS1}{pcr} {5} % tofu for blank, interrobang/down, ohm
\DeclareEncodingSubset{TS1}{phv} {5} % tofu for blank, interrobang/down, ohm
\DeclareEncodingSubset{TS1}{pnc} {5} % tofu for blank, interrobang/down, ohm
\DeclareEncodingSubset{TS1}{pplj}{5} % tofu for blank
\DeclareEncodingSubset{TS1}{pplx}{5} % tofu for blank
\DeclareEncodingSubset{TS1}{ppl} {5} % tofu for blank interrobang/down
\DeclareEncodingSubset{TS1}{ptm} {5} % tofu for blank, interrobang/down, ohm
\DeclareEncodingSubset{TS1}{pzc} {5} % tofu for blank, interrobang/down, ohm
\DeclareEncodingSubset{TS1}{ul9} {5} % tofu for blank, interrobang/down, ohm
\DeclareEncodingSubset{TS1}{dayroms}{6} % tofu for blank, interrobang/down, ohm
\DeclareEncodingSubset{TS1}{dayrom} {6} % tofu for blank, interrobang/down, ohm
\DeclareEncodingSubset{TS1}{augie}{8} % really only missing euro
\DeclareEncodingSubset{TS1}{put}  {8}
\DeclareEncodingSubset{TS1}{uag}  {8} % probably (currently broken distrib)
\DeclareEncodingSubset{TS1}{ugq}  {8}
\DeclareEncodingSubset{TS1}{zi4}     {9}
\DeclareEncodingSubset{TS1}{hls}     {5}
\DeclareEncodingSubset{TS1}{hlst}    {5}
\DeclareEncodingSubset{TS1}{hlct}    {5}
\DeclareEncodingSubset{TS1}{hlh}     {5}
\DeclareEncodingSubset{TS1}{hlx}     {8}
\DeclareEncodingSubset{TS1}{hlce}    {8}
\DeclareEncodingSubset{TS1}{hlcn}    {8}
\DeclareEncodingSubset{TS1}{hlcw}    {8}
\DeclareEncodingSubset{TS1}{hlcf}    {8}
\DeclareEncodingSubset{TS1}{lato-*}       {0}  % with a bunch of tofu inside
\DeclareEncodingSubset{TS1}{opensans-*}   {0}  % with a bunch of tofu inside
\DeclareEncodingSubset{TS1}{cantarell-*}  {0}  % with a bunch of tofu inside
\DeclareEncodingSubset{TS1}{tli}          {1}  % with lots of tofu inside
\DeclareEncodingSubset{TS1}{fbb-*}        {2}  % missing centoldstyle
\DeclareEncodingSubset{TS1}{Alegreya-*}               {2}
\DeclareEncodingSubset{TS1}{AlegreyaSans-*}           {2}
\DeclareEncodingSubset{TS1}{BaskervilleF-*}           {2}
\DeclareEncodingSubset{TS1}{DejaVuSans-TLF}           {2}
\DeclareEncodingSubset{TS1}{DejaVuSansCondensed-TLF}  {2}
\DeclareEncodingSubset{TS1}{DejaVuSansMono-TLF}       {2}
\DeclareEncodingSubset{TS1}{EBGaramond-*}             {2}
\DeclareEncodingSubset{TS1}{Merriwthr-OsF}            {2}
\DeclareEncodingSubset{TS1}{MerriwthrSans-OsF}        {2}
\DeclareEncodingSubset{TS1}{Montserrat-*}             {2}
\DeclareEncodingSubset{TS1}{MontserratAlternates-*}   {2}
\DeclareEncodingSubset{TS1}{NotoSansMono-TLF}         {2}
\DeclareEncodingSubset{TS1}{NotoSansMono-TOsF}        {2}
\DeclareEncodingSubset{TS1}{Tempora-TLF}              {2}
\DeclareEncodingSubset{TS1}{Tempora-TOsF}             {2}
\DeclareEncodingSubset{TS1}{XCharter-TLF}             {2}
\DeclareEncodingSubset{TS1}{XCharter-TOsF}            {2}
\DeclareEncodingSubset{TS1}{erewhon-*}                {2}
\DeclareEncodingSubset{TS1}{Arimo-TLF}                {3}
\DeclareEncodingSubset{TS1}{Crlt-*}                   {3}
\DeclareEncodingSubset{TS1}{IBMPlexMono-TLF}          {3}
\DeclareEncodingSubset{TS1}{IBMPlexSans-TLF}          {3}
\DeclareEncodingSubset{TS1}{IBMPlexSerif-TLF}         {3}
\DeclareEncodingSubset{TS1}{SourceCodePro-TLF}        {3}
\DeclareEncodingSubset{TS1}{SourceCodePro-TOsF}       {3}
\DeclareEncodingSubset{TS1}{SourceSansPro-*}          {3}
\DeclareEncodingSubset{TS1}{SourceSerifPro-*}         {3}
\DeclareEncodingSubset{TS1}{Tinos-TLF}                {3}
\DeclareEncodingSubset{TS1}{AccanthisADFStdNoThree-LF}{4}
\DeclareEncodingSubset{TS1}{Cabin-TLF}                {4}
\DeclareEncodingSubset{TS1}{Caladea-TLF}              {4}
\DeclareEncodingSubset{TS1}{Chivo-*}                  {4}
\DeclareEncodingSubset{TS1}{ClearSans-TLF}            {4}
\DeclareEncodingSubset{TS1}{Coelacanth-LF}            {4}
\DeclareEncodingSubset{TS1}{CrimsonPro-*}             {4}
\DeclareEncodingSubset{TS1}{FiraMono-TLF}             {4}
\DeclareEncodingSubset{TS1}{FiraMono-TOsF}            {4}
\DeclareEncodingSubset{TS1}{FiraSans-*}               {4}
\DeclareEncodingSubset{TS1}{Go-TLF}                   {4}
\DeclareEncodingSubset{TS1}{GoMono-TLF}               {4}
\DeclareEncodingSubset{TS1}{InriaSans-*}              {4}
\DeclareEncodingSubset{TS1}{InriaSerif-*}             {4}
\DeclareEncodingSubset{TS1}{LibertinusSans-*}         {4}
\DeclareEncodingSubset{TS1}{LibertinusSerif-*}        {4}
\DeclareEncodingSubset{TS1}{LibreBodoni-TLF}          {4}
\DeclareEncodingSubset{TS1}{LibreFranklin-TLF}        {4}
\DeclareEncodingSubset{TS1}{LinguisticsPro-LF}        {4}
\DeclareEncodingSubset{TS1}{LinguisticsPro-OsF}       {4}
\DeclareEncodingSubset{TS1}{LinuxBiolinumT-*}         {4}
\DeclareEncodingSubset{TS1}{LinuxLibertineT-*}        {4}
\DeclareEncodingSubset{TS1}{MintSpirit-*}             {4}
\DeclareEncodingSubset{TS1}{MintSpiritNoTwo-*}        {4}
\DeclareEncodingSubset{TS1}{PTMono-TLF}               {4}
\DeclareEncodingSubset{TS1}{PTSans-TLF}               {4}
\DeclareEncodingSubset{TS1}{PTSansCaption-TLF}        {4}
\DeclareEncodingSubset{TS1}{PTSansNarrow-TLF}         {4}
\DeclareEncodingSubset{TS1}{PTSerif-TLF}              {4}
\DeclareEncodingSubset{TS1}{PTSerifCaption-TLF}       {4}
\DeclareEncodingSubset{TS1}{Raleway-TLF}              {4}
\DeclareEncodingSubset{TS1}{Raleway-TOsF}             {4}
\DeclareEncodingSubset{TS1}{Roboto-*}                 {4}
\DeclareEncodingSubset{TS1}{RobotoMono-TLF}           {4}
\DeclareEncodingSubset{TS1}{RobotoSlab-TLF}           {4}
\DeclareEncodingSubset{TS1}{Rosario-*}                {4}
\DeclareEncodingSubset{TS1}{SticksTooText-*}          {4}
\DeclareEncodingSubset{TS1}{UniversalisADFStd-LF}     {4}
\DeclareEncodingSubset{TS1}{Almndr-OsF}               {5}
\DeclareEncodingSubset{TS1}{Baskervaldx-*}            {5}
\DeclareEncodingSubset{TS1}{Bttr-TLF}                 {5}
\DeclareEncodingSubset{TS1}{Cinzel-LF}                {5}
\DeclareEncodingSubset{TS1}{CinzelDecorative-LF}      {5}
\DeclareEncodingSubset{TS1}{Cochineal-*}              {5}
\DeclareEncodingSubset{TS1}{DejaVuSerif-TLF}          {5}
\DeclareEncodingSubset{TS1}{DejaVuSerifCondensed-TLF} {5}
\DeclareEncodingSubset{TS1}{GilliusADF-LF}            {5}
\DeclareEncodingSubset{TS1}{GilliusADFCond-LF}        {5}
\DeclareEncodingSubset{TS1}{GilliusADFNoTwo-LF}       {5}
\DeclareEncodingSubset{TS1}{GilliusADFNoTwoCond-LF}   {5}
\DeclareEncodingSubset{TS1}{OldStandard-TLF}          {5}
\DeclareEncodingSubset{TS1}{PlyfrDisplay-TLF}         {5}
\DeclareEncodingSubset{TS1}{PlyfrDisplay-TOsF}        {5}
\DeclareEncodingSubset{TS1}{TheanoDidot-TLF}          {5}
\DeclareEncodingSubset{TS1}{TheanoDidot-TOsF}         {5}
\DeclareEncodingSubset{TS1}{TheanoModern-TLF}         {5}
\DeclareEncodingSubset{TS1}{TheanoModern-TOsF}        {5}
\DeclareEncodingSubset{TS1}{TheanoOldStyle-TLF}       {5}
\DeclareEncodingSubset{TS1}{TheanoOldStyle-TOsF}      {5}
\DeclareEncodingSubset{TS1}{charssil-TLF}             {5}
\DeclareEncodingSubset{TS1}{Crimson-TLF}              {6}
\DeclareEncodingSubset{TS1}{LibertinusSerifDisplay-LF}{6}
\DeclareEncodingSubset{TS1}{LinuxLibertineDisplayT-*} {6}
\DeclareEncodingSubset{TS1}{LinuxLibertineMonoT-LF}   {6}
\DeclareEncodingSubset{TS1}{LinuxLibertineMonoT-TLF}  {6}
\DeclareEncodingSubset{TS1}{Ovrlck-LF}                {6}
\DeclareEncodingSubset{TS1}{ComicNeue-TLF}            {7}
\DeclareEncodingSubset{TS1}{ComicNeueAngular-TLF}     {7}
\DeclareEncodingSubset{TS1}{CormorantGaramond-*}      {7}
\DeclareEncodingSubset{TS1}{Heuristica-TLF}           {7}
\DeclareEncodingSubset{TS1}{Heuristica-TOsF}          {7}
\DeclareEncodingSubset{TS1}{IMFELLEnglish-TLF}        {7}
\DeclareEncodingSubset{TS1}{LibreBskrvl-TLF}          {7}
\DeclareEncodingSubset{TS1}{LibreCsln-*}              {7}
\DeclareEncodingSubset{TS1}{Lbstr-LF}                 {7}
\DeclareEncodingSubset{TS1}{Mrcls-LF}                 {7}
\DeclareEncodingSubset{TS1}{NotoSans-*}               {7}
\DeclareEncodingSubset{TS1}{NotoSerif-*}              {7}
\DeclareEncodingSubset{TS1}{Quattro-LF}               {7}
\DeclareEncodingSubset{TS1}{QuattroSans-LF}           {7}
\DeclareEncodingSubset{TS1}{Frm-LF}                   {7}  % the superiors are missing
\DeclareEncodingSubset{TS1}{LibertinusMono-TLF}       {8}
\DeclareEncodingSubset{TS1}{AlgolRevived-TLF}         {9}

[cfr42]

[Which is not the sources of 2e, but is source as opposed to documentation.]

[FrankMittelbach]

@cfr42 latexrelease is not a (real) package, even though it has the .sty extension. It is the roll back mechanism of the LaTeX kernel,. So what you see is code to roll back or in some cases roll forward. To be used as \RequirePackage... and yes it contains these defaults that really shouldn't be in the kernel (or in this roll back code) but in the individual .fd files.

[FrankMittelbach]

Are you using the original fontinst of the tool my Marc?

Sorry. I don't understand this question.

author of "autoinst", that's what Bob and I think Michael are using for their font packages. Anyway, I think you answered that you using a lua script

The reason I don't want to put them in the .fd files is that the declarations are much easier to override if they are in the .sty files.

agreed, if the setting is wrong (which it shouldn't be if it is set by the font bundle maintainer) then it is slightly more convoluted to get it right as it has to be done after the .fd got loaded, but that's the same for any setting made in the .fd file and the solution then would be

\LoadFontDefinitionFile{ts1somefamily}
\DeclareEncodingSubset{TS1}{somefamily}{7}

so not too bad.

[cfr42]

@cfr42 latexrelease is not a (real) package, even though it has the .sty extension. It is the roll back mechanism of the LaTeX kernel,. So what you see is code to roll back or in some cases roll forward. To be used as \RequirePackage... and yes it contains these defaults that really shouldn't be in the kernel (or in this roll back code) but in the individual .fd files.

I figured that out later, but my point was just that if you grep for declarations in base, you get a lot of examples in .sty files and none in .fd files. In general, following the pattern in base is 'safe'. So whether it is a 'real' package or not isn't really relevant. I'm just trying to explain why it makes perfect sense to me that people have put the declarations in .sty files. These were the examples I found when I looked to see how things were supposed to work.

[cfr42]

author of "autoinst", that's what Bob and I think Michael are using for their font packages. Anyway, I think you answered that you using a lua script

Oh, OK. Almost all of my font packages use fontinst because I find it easier to control than autoinst. berenisadf uses GNU make to run a bunch of scripts, but the .fd files are handwritten in that case. fontinst.lua doesn't do anything interesting. It just integrates whichever tool into l3build, edits some of the generated files if needed, automates some lvt/tex creation and alters what ctan does. It just automates the process for convenience. The font setup is all done by fontinst or ff/afm2pl or whatever.

[josephwright]

@cfr42 Based on our other conversation, I wonder if we could look at having a generic version of your fontinst that would be set up to integrate with l3build and be useable by font people generally?

[cfr42]

@cfr42 Based on our other conversation, I wonder if we could look at having a generic version of your fontinst that would be set up to integrate with l3build and be useable by font people generally?

It is very messy and I don't even know how to isolate the process properly.

[cfr42]

@josephwright in theory, it is somewhat generic already in that you can adapt it to use whatever font creation recipe you want (which is what berenisadf's build.lua does). but whether it actually works at all anywhere else, I don't know.

[cfr42]

@cfr42 Based on our other conversation, I wonder if we could look at having a generic version of your fontinst that would be set up to integrate with l3build and be useable by font people generally?

So I now think this is possibly possible.

[cfr42]

https://codeberg.org/cfr/nfssext/issues/9

[dr-scsi]

Apologies if my comment is off, but IIUC, putting \DeclareEncodingSubset{TS1}{somefamily}{7} in the .fd means that the compilation breaks if the .fd is loaded in the body of the document and not in the preamble. Example: Save this snippet as ts1anonymouspro.fd in a directory:

\ProvidesFile{ts1anonymouspro.fd}
   [2024/11/19 Fontinst v1.933 font definitions for TS1/AnonymousPro.]

\expandafter\ifx\csname ANP@scale\endcsname\relax
   \let\ANP@@scale\@empty
\else
   \edef\ANP@@scale{s*[\csname ANP@scale\endcsname]}%
\fi

\DeclareFontFamily{TS1}{AnonymousPro}{}

\DeclareFontShape{TS1}{AnonymousPro}{m}{n}{
   <->\ANP@@scale AnonymousPro-Regular-ts1
}{}

\DeclareFontShape{TS1}{AnonymousPro}{m}{sc}{
   <->\ANP@@scale AnonymousPro-Regular-ts1
}{}

\DeclareFontShape{TS1}{AnonymousPro}{m}{it}{
   <->\ANP@@scale AnonymousPro-Italic-ts1
}{}

\DeclareFontShape{TS1}{AnonymousPro}{b}{n}{
   <->\ANP@@scale AnonymousPro-Bold-ts1
}{}

\DeclareFontShape{TS1}{AnonymousPro}{b}{sc}{
   <->\ANP@@scale AnonymousPro-Bold-ts1
}{}

\DeclareFontShape{TS1}{AnonymousPro}{b}{it}{
   <->\ANP@@scale AnonymousPro-BoldItalic-ts1
}{}

\DeclareFontShape{TS1}{AnonymousPro}{m}{sl}{<->ssub * AnonymousPro/m/it}{}
\DeclareFontShape{TS1}{AnonymousPro}{bx}{n}{<->ssub * AnonymousPro/b/n}{}
\DeclareFontShape{TS1}{AnonymousPro}{bx}{sc}{<->ssub * AnonymousPro/b/sc}{}
\DeclareFontShape{TS1}{AnonymousPro}{bx}{it}{<->ssub * AnonymousPro/b/it}{}
\DeclareFontShape{TS1}{AnonymousPro}{b}{sl}{<->ssub * AnonymousPro/b/it}{}
\DeclareFontShape{TS1}{AnonymousPro}{bx}{sl}{<->ssub * AnonymousPro/b/sl}{}

\DeclareEncodingSubset{TS1}{AnonymousPro}{0}

\endinput

And save this snippet in the same directory:

\listfiles
\documentclass[a4paper,10pt]{article}
\begin{document}
\usefont{TS1}{AnonymousPro}{m}{n}
\end{document}

and run pdflatex on it which says:

! LaTeX Error: Can be used only in preamble.

See the LaTeX manual or LaTeX Companion for explanation.
Type  H <return>  for immediate help.
 ...

l.49 \DeclareEncodingSubset
                           {TS1}{AnonymousPro}{0}

This is because of the following code in lttextcomp.dtx:

% Any reason to allow those in the middle of documents?
\@onlypreamble\DeclareEncodingSubset
\@onlypreamble\DeclareEncodingSubset@aux
\@onlypreamble\@DeclareEncodingSubset

IIUC, the "only preamble" restriction must be lifted before \DeclareEncodingSubset can go into a .fd file.

[cfr42]

@dr-scsi Note that I don't think the result of the test is reliable if the fonts use fontinst.

[dr-scsi]

@cfr42 - You mean the result of latex checkencodingsubset.tex FONTNAME, or am I missing something?

[FrankMittelbach]

Apologies if my comment is off, but IIUC, putting \DeclareEncodingSubset{TS1}{somefamily}{7} in the .fd means that the compilation breaks if the .fd is loaded in the body of the document and not in the preamble.

how embarrassing :-( ... I lobby for the change (because it is the right thing imho) and write articles and newsletters about it and then it doesn't actually work because of an oversight ... yes the \@onlypreamble dreclarations have to be removed from the kernel and this will in fact happen in a hotfix so we will have a PL1 soon for the current release

[cfr42]

@FrankMittelbach Are you aware checkencodingsubset often (always?) recommends 0 for fontinst installed fonts?

[FrankMittelbach]

If fontinst does add tofu glyphs for those that do not exist then this wouldn't be surprising (but unfortunate). Is that the reason for it? Or saying it differently, can you give me an example name so that I can see what is happening?

[cfr42]

If fontinst does add tofu glyphs for those that do not exist then this wouldn't be surprising (but unfortunate). Is that the reason for it?

Yes. Aside from the fact tofu is fantastic stuff, fontinst records the characters as missing and substitutes dead meat. For example, from one of the virtual fonts (ylyr8c):

(CHARACTER D 25 (COMMENT arrowright)
   (CHARWD R 0.5)
   (CHARHT R 0.500)
   (MAP
      (SETRULE R 0.5 R 0.5)
      (SPECIAL Warning: missing glyph `arrowright')
      )
   )

So it is possible to tell what is missing from the log fontinst produces or from the virtual fonts it writes, but later LaTeX doesn't see anything wrong. And you probably cannot easily tell even from the vfs because, if I remember correctly, the conversion process strips comments, so you can't just convert them back to text and grep. You'd have to deduce it from the construction of corpses. (The excerpt above is from a vpl written by fontinst.)

So I think you need to qualify your recommendation. Otherwise packages which currently fallback to subset 9 may suddenly produce black boxes if their authors use fontinst and take your advice.

Of course, ideally, fontinst would be updated to not do this, but I'm not sure how realistic that is.

[FrankMittelbach]

So I think you need to qualify your recommendation. Otherwise packages which currently fallback to subset |9| may suddenly produce black boxes if their authors use |fontinst| and take your advice.

I already qualified that (at least in the newsletter, that there are fonts which contain tofu (yes good stuff :-) if used in the kitchen). What I wasn't aware of, is that it is a general "feature" for fonts produced with fontinst, so that recommendation could and should be improved. Not sure, but I wonder if checkencodingsubset could (and should) test if there are chars with 5x5 size and in that case perhaps issue a warning that there might be tofu inside.

[cfr42]

Not sure, but I wonder if checkencodingsubset could (and should) test if there are chars with 5x5 size and in that case perhaps issue a warning that there might be tofu inside.

fontinst fds provide LaTeX with the information about the producer directly:

\ProvidesFile{ts1yly.fd}
   [2024/09/27 Fontinst v1.933 font definitions for TS1/yly.]

so perhaps the test should check for that?

[FrankMittelbach]

maybe, the question is which has more false positives (or misses tofu that is inside).

[cfr42]

@cfr42 - You mean the result of latex checkencodingsubset.tex FONTNAME, or am I missing something?

@dr-scsi Yes. The result is always a recommendation of 0 regardless. I think this might not happen if you use sufficiently customised inputs, but you'd have to avoid settings in various .mtxs.

[cfr42]

maybe, the question is which has more false positives (or misses tofu that is inside).

Well, if you check for 5x5 rules, you'll miss some missing glyphs,

(CHARACTER D 27 (COMMENT tieaccentcapital)
   (CHARWD R 0.25)
   (CHARHT R 0.652)
   (MAP
      (MOVEUP R 0.504)
      (SETRULE R 0.148 R 0.25)
      (SPECIAL Warning: missing glyph `tieaccentcapital')
      )
   )

This is from a vpl produced for a font from gentium-tug. textcomp.mtx uses a different definition for rules when accents are missing.

[cfr42]

Technically, it would be easy to stop fontinst doing this .... You'd just need a drop in mtx before textcomp.mtx is loaded. But that would require all package authors to update all driver files and regenerate the fonts. The first could be scripted easily, but regenerating fonts is a lot to ask.

[FrankMittelbach]

Technically, it would be easy to stop fontinst doing this .... You'd just need a drop in mtx before textcomp.mtx is loaded. But that would require all package authors to update all driver files and regenerate the fonts. The first could be scripted easily, but regenerating fonts is a lot to ask.

On the other hand, putting tofu into fonts may have looked like a good idea back in the days when Alan and Sebastian produced the early versions of fontinst (or whoever was involved then) but it is not a good idea at all for the users, because in contrast of getting a missing glyph warning or error (depending on your setting) you only find out if you carefully scan your document visually. So yes, it is extra work for the font maintainers, but it would be in the interest of their users --- and it is somewhat unrelated to the discussion here, except that if the tofu is not there LaTeX could do corrective actions (assuming the subset information is correct for the font).

[FrankMittelbach]

maybe, the question is which has more false positives (or misses tofu that is inside).

Well, if you check for 5x5 rules, you'll miss some missing glyphs,

(CHARACTER D 27 (COMMENT tieaccentcapital)
   (CHARWD R 0.25)
   (CHARHT R 0.652)
   (MAP
      (MOVEUP R 0.504)
      (SETRULE R 0.148 R 0.25)
      (SPECIAL Warning: missing glyph `tieaccentcapital')
      )
   )

This is from a vpl produced for a font from gentium-tug. textcomp.mtx uses a different definition for rules when accents are missing.

ok, so if this varies from font to font then it is not a good idea to heuristically deduce tofu from glyph dimensions. So checking if the file info mentions fontinst and then put out a general warning might be the better choice.

[cfr42]

ok, so if this varies from font to font then it is not a good idea to heuristically deduce tofu from glyph dimensions.

Assuming use of textcomp.mtx and no prior definition of \unfakable or \unfakableaccent, it uses

\setcommand\unfakable#1{
   \setglyph{#1}
      \glyphrule{500}{500}
      \glyphwarning{missing glyph `#1'}
   \endsetglyph
}
% \unfakableaccent is similar, but has an ``accent position''
\setcommand\unfakableaccent#1{
   \setglyph{#1}
      \moveup{\add{\int{xheight}}{50}}
      \movert{\scale{\add{\int{xheight}}{50}}{\int{italicslant}}}
      \glyphrule{250}{\sub{\sub{\int{ascender}}{\int{xheight}}}{100}}
      \movert{\neg{\scale{\add{\int{xheight}}{50}}{\int{italicslant}}}}
      \glyphwarning{missing glyph `#1'}
   \endsetglyph
}

But there are several ways in which these could fail to actually be the active definitions when the vpls are generated for TS1. They aren't likely, but even ruling those out, you've got some missing glyphs where the rule size depends on the font.

[cfr42]

Technically, it would be easy to stop fontinst doing this .... You'd just need a drop in mtx before textcomp.mtx is loaded. But that would require all package authors to update all driver files and regenerate the fonts. The first could be scripted easily, but regenerating fonts is a lot to ask.

On the other hand, putting tofu into fonts may have looked like a good idea back in the days when Alan and Sebastian produced the early versions of fontinst (or whoever was involved then) but it is not a good idea at all for the users, because in contrast of getting a missing glyph warning or error (depending on your setting) you only find out if you carefully scan your document visually. So yes, it is extra work for the font maintainers, but it would be in the interest of their users --- and it is somewhat unrelated to the discussion here, except that if the tofu is not there LaTeX could do corrective actions (assuming the subset information is correct for the font).

Technically, it is unrelated. But, in reality, if you wanted people to regenerate fonts, you might need to tell them that's possible. That would mean either telling them to use an updated version of fontinst or telling them how to avoid the gravestones in TS1. This isn't a LaTeX bug/problem, but neither is the issue you're addressing in ltxnews. (Except it's a problem insofar as you have definitions in the kernel, which doesn't even apply to a number of font packages in TL.)

Which is to say, it seemed relevant to point out that this feature/bug can be worked around quite easily without touching fontinst code, if getting that updated proves problematic. For TS1 it turns out to be trivial. My workaround doesn't work for T1 right now and I've not yet looked into why not. (Not really surprising given the additional complexity in generating text versus symbol fonts.)

[cfr42]

On the other hand, putting tofu into fonts may have looked like a good idea back in the days when Alan and Sebastian produced the early versions of fontinst (or whoever was involved then) but it is not a good idea at all for the users, because in contrast of getting a missing glyph warning or error (depending on your setting) you only find out if you carefully scan your document visually.

I think the idea was that users would also get a warning about the missing glyphs. Although comments are stripped, the SPECIAL Warning lines survive conversion to binary. Going by the comments, I think these were expected to result in warnings being generated each time the glyph was used. But this does not seem to be the case for pdfTeX, at least. But I don't think the code was written with pdfTeX in sight ;). I think the rules are just meant to be an additional aid to finding the location of the problems those warnings are meant to identify.

[FrankMittelbach]

As I said, looked like a good idea back then and probably was, with dvi -> dvips -> PS, but with pdftex and with nowadays generating an error or at least a log entry if a glyph is missing it isn't any longer.

[cfr42]

[Edit: yes, it does. Please ignore.]

[cfr42]

How is the * shortcut supposed to be used when declaring encoding subsets? source2e says,

As these days many font families are set up to end in -LF (lining figures), -OsF
(oldstyle figures), etc. the declaration supports a shortcut: if the font family name ends
in -* then the star gets replaced by these common ending, e.g.,

\DeclareEncodingSubset{TS1}{Alegreya-*}{2}

is the same as writing

\DeclareEncodingSubset{TS1}{Alegreya-LF}{2}
\DeclareEncodingSubset{TS1}{Alegreya-OsF}{2}
\DeclareEncodingSubset{TS1}{Alegreya-TLF}{2}
\DeclareEncodingSubset{TS1}{Alegreya-TOsF}{2}

These are setup as distinct families, each with its own .fd:

tex/latex/alegreya/TS1Alegreya-LF.fd  tex/latex/alegreya/TS1Alegreya-OsF.fd  tex/latex/alegreya/TS1Alegreya-TLF.fd  tex/latex/alegreya/TS1Alegreya-TOsF.fd

Moreover, as far as I know, they pretty much have to be set up as distinct families and each family has to have its own .fd if LaTeX is to load them correctly.

So I suspect the -* shortcut needs to be reconsidered but, if not, perhaps the documentation could get an update to explain its intended use?

I suspect this also encouraged me to think the declarations should go in .sty files. (I may not have read ltxnews, but I certainly read this part of source2e.)

[cfr42]

... except that if the tofu is not there LaTeX could do corrective actions (assuming the subset information is correct for the font).

Perhaps I've misunderstood, but I thought the gravestones were a problem when determining the encoding. LaTeX then substitutes according to the declared subset. So people don't really need to regenerate the fonts, if they don't want to. It should work just as well if they add the declarations post hoc. Admittedly, they'll have to remember if they later update the fonts, so an automated way to generate the fds would still be useful, but they don't really need to do that just to get LaTeX to substitute in TS1.

So you could provide something like the code @davidcarlisle posted or a script to extract specials from the virtual fonts. Or just a file to process with latex > dvips. Or whatever.