Snapshot testing PDFs

[rythos42]

The Problem
I'm trying to do some really quick snapshot testing of PDFs. Our system generates them using Spire.PDF and sends them to DocuSign encoded in base64. My snapshots take the generated base64 and compare it against "known good" to see if anything changed.

This process has been, so far, 100% reproducible on my computer and a co-workers computer. But when we put the tests up on Jenkins, the tests fail. Looking deeper into the problem, the same test cases with the same base documents generate completely different bytes, while also generating identical looking PDFs.

Some Details
Or things I've ruled out. Or other thoughts.

• At some point in the process I have to convert to string and back, and I've checked that the encoding on my computer and Jenkins is the same.
• I've replaced any "random" meta data values like created date and ID in the PDF with static values so snapshots are possible.
• I'm open to other ideas of snapshot testing that can be done quickly! I'm looking for a "quick win" to reduce the amount of regression testing we need to do every release on this stuff, and not a "comprehensive testing solution".
• The build environment is destroyed every build and recreated, while my computer is not.
• I haven't checked that the versions of Spire are the same (because it's a weekend and I don't have dev ops permissions and politics...), but both processes are getting the library using NuGet so I don't have any reason why they might be different.
• We're licensing Spire using the embedded resource method, and neither generated PDF implies it is a trial version.

Thanks!

[rythos42]

I tried a new thought -- to use ExtractText to build up a StringBuilder of the text contents and compare that. I was surprised to note that the text contents always seemed to contain the words "Evaluation Warning", regardless of whether my license was installed or not.

I'm also unable to confirm that the license is installed in any scenario. None of the PDFs output give indication that they were built using the evaluation version, and we aren't using the free version, regardless of whether I put the license in the assembly or build a new project that doesn't have the license installed and try that.


My theory is that Jenkins somehow has the license installed, but my local does not. So my snapshots pass locally because both the snapshot and the test are evaluations, but Jenkins has the license so maybe it generates evaluation-free PDFs and compares against my trial PDF snapshot? It's a guess that I can't confirm because I'm unable to build the negative case (where the PDFs show Evaluation Warning).

(seems like a weird problem to have though...)

[Amber.Gu]

Hi,

Thanks for your inquiry. Sorry to reply late for weekend.

For the first issue, to help us better investigate it, could you please offer us the following information?
1. Your input files(if any).
2. Your result file.
3. The complete code you were using which could reproduce your issue directly.
4. The OS and Region information, e.g. Win7 64bit, China/Chinese.
You could upload them here or send us(support@e-iceblue.com) via email.

For the second issue, kindly note that the license file you bought is for Spire.Doc, which means it is only licensed to Spire.Doc. If you want to remove the warning information of Spire.PDF, you will need the license file of Spire.PDF. Our sales team just sent you a one-month free license of Spire.PDF to help you remove the warning information. Please check and apply it to have a better evaluation on our product. Here is a guide for you: https://www.e-iceblue.com/Tutorials/Licensing/Licensing.html.

Best wishes,
Amber
E-iceblue support team

[rythos42]

Thanks very much! I've decided to go a different route with this, and it is working well enough for my purposes.

[Amber.Gu]

Hi,

Thanks for your reply. Glad to hear that you have managed to solve this issue yourself.
If there is any question in the future, welcome to contact us. Have a nice day.

Best wishes,
Amber
E-iceblue support team