Layered PDFs in Scribus, potential fix!

[Christopher Roe]

I recall a while back that Sirrob, Jabbro, and some other guys were getting all kinds of headache from a Scribus bug that affected layered PDFs, and I've found a potential fix.

The problem is Scribus doesn't support all 3 visibility flags for printing layers, only the flags that specify layers are always printed or never printed. Jabbro filed a bug report years ago, but the Scribus devs basically ignored it.

The fix is simple, and it was actually mentioned cryptically in the same bug report, which is what piqued my interest and got me to dig into the PDF file format reference for details. Here's all you have to do to make layers work properly.

1. Export your layered document to PDF as normal, just like you were doing before.

2. Open the PDF in Notepad++ or another text editor capable of handling large files.

3. CTRL-F and search for all lines that begin with "/Usage" (without the quotes), and delete them. Here's an example of what one of those lines looks like:

"/Usage <</Print <</PrintState /ON>> /View <</ViewState /ON>>>>>>"

That's all you have to do, just delete all lines that look like that, then save the file and open it in Acrobat Reader.

Okay, I'd like Sirrob and anyone else who was having trouble with layered PDFs to try this and let me know if it works for them too.

[Sirrob01]

How do I buy you a Beer or your favorite alcoholic beverage Chris?

I did a quick test and it seems to work perfectly on my quick 3 layer pdf attached.

I'll try this more fully when I layout the next Hoard (& Steammech) as it'll save me issues with rego marks for cutters.

Attachments:

[Christopher Roe]

YES! I'm glad it works for somebody else. I was a little suspicious because it was so easy that it almost seemed too good to be true.

No need to buy me a beer. I'm just glad I was able to help solve the problem.

[nikloveland]

Oh wow! That is quite the impressive hacking. Maybe the Scribus team should be notified or at least added to the bug report.

[Christopher Roe]

Oh, they know, fixing it's just not a priority for them. In the meantime, this workaround reduces it from a showstopping bug to a minor inconvenience.

[tonsha]

I tried creating a simple layered file using Portable Scribus 1.3.3.9 in Windows - but the pdf file didn't have any '/Usage' strings in it. Anywhere.

I'm wondering:

(a) whether or not I created the layers correctly - any chance of a tutorial?

(b) whether v1.3.3.9 is the 'correct' version?

But it is exciting that this could solve a major problem in linux. YAY!

DaveA

[Sirrob01]

Tonsha quick things I can think off.

Make sure you turn on the layers window and add different content to different layers (Window>>Layers). Click on the layer you want then use insert>>"stuff" to add text and images.

when you create the pdf ensure you tick on the include layers tickbox on the dialogue box that pops up and set compatibility to Acrobat 6 (I'm not sure if 1.3.3.9 supports acrobat 6 but probably)

Hope the above is enough to point you in the right direction to get you started

[Christopher Roe]

Yeah, if you don't see the /Usage strings, then the layers likely weren't exported in the PDF.

Try exactly what Rob suggested--make sure the layers window is open. Add/delete layers with the + and - buttons, sort them with the up and down arrow buttons, click on a layer to make it the active layer, and anything you insert into or draw on the pages will be assigned to that layer. Once you export to PDF, make sure you choose PDF 1.5 or 1.6 at minimum and that you tick the checkbox for keeping the layers.

[tonsha]

Oh wow! I hadn't spotted your replies earlier. Sorry gents.

I've just tried it - Sirrob's suggestion about to 'include layers' when creating the PDF (I had to switch to PDF 1.5 to make that active - DUH!) - and it worked perfectly.

I didn't need to mess about with the PDF file either. It 'just worked' in Acrobat Reader X. I'll have to try it on Linux as well though.

Absolutely brilliant!!

Chris, Rob - thanks for the quick responses. What a fantastic utility! (And opportunity...!)



DaveA

[Sirrob01]

Tonsha the layering will work okay its when you got to print that you'll strike problems and the fix that Chris details above is needed.

To test it just hide a layer normally visible and send it to a pdf printer (saves paper) and your should be invisible layer will be printed visible .

[tonsha]

Jan 15, 2013 21:24:11 GMT -9 Sirrob01 said:

Tonsha the layering will work okay its when you got to print that you'll strike problems and the fix that Chris details above is needed.

To test it just hide a layer normally visible and send it to a pdf printer (saves paper) and your should be invisible layer will be printed visible .

OK - I've finally got into work and cleared space to try this out...and it 'just worked'! I'm a bit gobsmacked myself.

I was able to print all 3 layers of my test PDF separately. I re-checked the file in notepad++ and there aren't any /Usage lines.

I've no idea why it worked straight off the bat - just lucky I guess!

I'll try it at home on Linux as well. That's where I do nearly of my miniature drawing anyway.

DaveA

[Sirrob01]

Settings used?? version?? I'd love to know

[papersmithforge]

I found this thread much the same as others I suspect, looking for a layer solution in Scribus. Applying the solution suggested by the original poster works. However, it didn't seem to work in Foxit Reader. I imagine it doesn't support OCGs very well. I've thus switched to PDF-XChange Viewer, also free, lots more features, no install, and lightweight. Furthermore, it does work properly with Scribus generated v1.5 PDFs with layers.

I didn't have a binary editor like Notepad++ installed on my system. Therefore, I went the route of writing my own shell executable to do the task. It's coded in Java, and takes the file's location as its parameter, it will output the file to the same location but renamed to the original filename with the text "_parsed" attached to it. It's very simple to use and if it doesn't work it will spit out a cryptic error message, OR if the document has no /Usage strings present within it, it will simply leave the original file unchanged.

This basically eliminates the need to open the file and do a Ctrl + F. You just run the file through this executable and done. It will work on Linux, Windows, Mac, or whichever OS runs Java 7. It ONLY eliminates the /Usage information for layers which are set as BOTH printable and viewable. If the author of the document makes some other combination that involves an off, it is assumed to be purposeful and deliberate, and those settings should be preserved (this is particular to my own usage of course).

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.InvalidPathException;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardOpenOption;
import java.util.Arrays;

public class ScribusParser {

public static void main(String[] args)
throws InvalidPathException, IOException, OutOfMemoryError, UnsupportedOperationException {

Path pdfPath = Paths.get(args[0]);

byte[] haystack = Files.readAllBytes(pdfPath);

pdfPath = Paths.get(
args[0].substring(
0, args[0].length() - 4).concat(
"_parsed.pdf"));

/*
* The following three cases are ignored because they are assumed to be
* intentional and/or possibly useful so their state should be preserved:
* /Usage <</Print <</PrintState /OFF>> /View <</ViewState /ON>>>>>>
* (ink saving, no printing allowed)
* /Usage <</Print <</PrintState /ON>> /View <</ViewState /OFF>>>>>>
* (watermark)
* /Usage <</Print <</PrintState /OFF>> /View <</ViewState /OFF>>>>>>
* (bookmark object)
*
*  Only the latter case should be controlled by the user for
*  multi-layer Scribus PDF documents (PDF v1.5 Acrobat 6)
*/
byte[] needle = "/Usage <</Print <</PrintState /ON>> /View <</ViewState /ON>>>>>>".getBytes();

int from = 0;
int to = 0;
while(-1 < (to = indexOf(haystack, needle, from))){
Files.write(
pdfPath,
Arrays.copyOfRange(haystack, from, to - 1),
StandardOpenOption.CREATE,
StandardOpenOption.APPEND,
StandardOpenOption.WRITE);
from = to + needle.length;
}
if(from != 0){
System.out.print("Successfully parsed as: ");
System.out.println(Files.write(
pdfPath,
Arrays.copyOfRange(haystack, from, haystack.length - 1),
StandardOpenOption.CREATE,
StandardOpenOption.APPEND,
StandardOpenOption.WRITE).toString());
}

System.exit(0);
}

public static int indexOf(byte[] haystack, byte[] needle){

return indexOf(haystack, needle, 0);
}

public static int indexOf(byte[] haystack, byte[] needle, int offset){

if(haystack.length == 0 || needle.length == 0)
return -1;

if(offset < 0)
offset = haystack.length + offset;

if(offset < 0)
offset = 0;

for(int i = offset; i <= haystack.length - needle.length; i++){
for(int j = 0; j < needle.length; j++){
if(needle[j] != haystack[i + j])
break;
if(j == needle.length - 1)
return i;
}
}

return -1;
}

}

Of course you'll need to know how to compile Java code, but for such a simplistic program there are plenty of tutorials out there on how to do so, and on how to execute the compiled *.class file as well.

[jeffgeorge]

As of September, 2016, I'm trying to create a layered PDF in Scribus with "Print when visible" layers, and I'm not having any luck with this workaround.

I'm exporting the layered file from Scribus, using PDF 1.5 with Include Layers checked. I open the .pdf file in Notepad++, delete the Usage lines, and resave the file. When I open the hacked .pdf in Acrobat Reader DC (2015 version 2015.017.20053), I can click the visibility of each layer ON or OFF, but the display of the file is unchanged when I set a layer to OFF, either on screen or on the printout.

I'm using Scribus v 1.5.2, although I've also tried the last 1.4 version, as well as the 1.5.3.svn version, and I'm getting the same results with each.

I've researched Jabbro's issue request with the Scribus developers, and while it appears that "Print when visible" is slated for introduction in 1.5.3, it shows up as a still-active (that is, not resolved) issue, and there is no evidence that a "Print when visible" option is available in the currently downloadable build of Scribus 1.5.3.

Is anyone having any luck with a "Print when visible" workaround using the current versions of Scribus and Acrobat Reader?

[mproteau (Paper Realms)]

Sept 19, 2016 18:06:22 GMT -9 jeffgeorge said:

As of September, 2016, I'm trying to create a layered PDF in Scribus with "Print when visible" layers, and I'm not having any luck with this workaround.

I'm exporting the layered file from Scribus, using PDF 1.5 with Include Layers checked. I open the .pdf file in Notepad++, delete the Usage lines, and resave the file. When I open the hacked .pdf in Acrobat Reader DC (2015 version 2015.017.20053), I can click the visibility of each layer ON or OFF, but the display of the file is unchanged when I set a layer to OFF, either on screen or on the printout.

I'm using Scribus v 1.5.2, although I've also tried the last 1.4 version, as well as the 1.5.3.svn version, and I'm getting the same results with each.

I've researched Jabbro's issue request with the Scribus developers, and while it appears that "Print when visible" is slated for introduction in 1.5.3, it shows up as a still-active (that is, not resolved) issue, and there is no evidence that a "Print when visible" option is available in the currently downloadable build of Scribus 1.5.3.

Is anyone having any luck with a "Print when visible" workaround using the current versions of Scribus and Acrobat Reader?

I use 1.4 all the time with success. I tried 1.5 and it didn't do so hot but I can't remember why. I uninstalled it and went back to 1.4.

[jeffgeorge]

Oh, wow...

Based on your suggestion, but without much hope, I went back and retraced my steps with Scribus 1.4, just in case I'd missed something. I reverted to the last version of my project saved in 1.4.6, since 1.5.X has a different file format (fair amount of work done in 1.5.2 lost, but oh well...), and carefully exported a layered PDF. Then I opened the file in Notebook++, deleted the usage lines, and resaved the file. I opened the file with the current version of Acrobat Reader and holy cow, it worked! All the layers are magically "Print when visible"!!!!

The layers toggle on and off on screen as they are supposed to, and print or don't print according the the current visibility setting. PERFECT!

I've been beating my head against this for about a week now, without being able to make the workaround work. I'm not afraid to tinker with the code in a PDF file, but it didn't seem to be working. All I can figure is that when I was still working with Scribus 1.4, I ONLY tried the \Usage-delete hack using Sublime Text 2 as my editor, not Notepad++. Somehow, saving the text version of the PDF file in Sublime broke the file so it wouldn't open. I then tried Scribus 1.5.2 and 1.5.3, still using Sublime for the hack, and getting the same broken file. (By "broken", I mean the fonts were getting scrambled, rendering most of the document blank.) It occurred to me that the people using the hack successfully were apparently all using Notepad++, so I tried that instead of Sublime, and although the layer hack didn't work, at least the Notepad++ didn't break the file.

It wasn't until your response, mproteau, that it occurred to me to try Scribus 1.4.6 with Notebook++, and that was the magic combination. (In fact, it hadn't occurred to me that I had not tried that combo until you pointed it out to me). Now I can actually create the multi-layered PDF that I always planned to. Thanks so much!

[berneart76]

I haven't used Scribus at all, but upon reading this thread I'd like to nominate Christopher Roe as hero of the week! 

[mproteau (Paper Realms)]

A little tidbit I just discovered. So, I've always hated my PDFs. When I close them in Acrobat Reader, I'm always prompted to save, even if I hadn't changed anything. This is a behavior brought on by the removal of these <<Usage...>> lines from the PDFs I export out of Scribus. Well, I finally decided to let Acrobat Reader save one of the PDFs after I had opened it, without changing anything in the file. I wanted to look at the resulting PDF to see what changed. It turns out, Reader ended up writing it out as a 1.6-format file (my Scribus version stops at 1.5). The contents of the file is very different, but the results *appear* to be the same. It looks like layers should continue to print fine, but now I'm no longer prompted to save my PDFs, even after toggling the visibility of layers!

Thought I'd mention it in case other people are relying on Scribus for making layered PDFs, and have maybe noticed the same thing.

[jeffgeorge]

I'm waking up this slumbering thread, because it's worth keeping alive. I have a bit of a warning, and a question about the state of Scribus:

The warning: Don't buy Ashampoo PDF Pro for minis
I just dropped $20 on a Humble Bundle package--which will expire before you read this, so don't worry about falling into the trap--that included Ashampoo PDF Pro 2. I bought the bundle mostly hoping that this program would be a cleaner and more reliable solution for creating the kind of layered PDFs we use to distributed printable miniatures. Well, it's not. It doesn't let you add or delete layers, and it doesn't let you set printability for layers. The editing interface looks more friendly Scribus, but it lacks the key feature we need, so its essentially useless to us. Save your money. (Hopefully, I'll get some use out of other utilities that were in the bundle, so my $20 won't be wasted. Still, it was a cheap bet, so I'm not too distressed.)

The question: What version of Scribus are we using now?
All the gyrations and workarounds mentioned in this thread from several years ago were developed when the current versions of Scribus were 1.4.6 (Stable) and 1.5.2 (Development). Now Scribus is at 1.4.8 and 1.5.4. Are the new versions any better than the ones previously mentioned? I was only able to get Christopher Roe workaround to work when specifically editing PDF files created with Scribus 1.4.6 using Notepad++. Does anyone know if it works with 1.4.8, and is there any other reason to update/upgrade? And does 1.5.4 just include the ability to set printable layers natively, eliminating the need for the workaround altogether?

I'll eventually experiment with the various versions and find out for myself, but I was wondering if anyone else had already checked them out.

A bonus question: Has anyone found a less expensive PDF option than Acrobat?
Has anyone found another PDF editor that allows you to control the layers the way we need to, without paying Adobe's brutal monthly fee? I'd be willing to spend up to $100 one time for an application that worked well, but I just don't create enough PDFs for pay to justify the cost of Creative Cloud.

Thanks, and everyone stay safe and healthy!