Weird behavior when using ExtractText with ExtractArea

Wed Feb 09, 2022 12:45 am

Hey there - I'm looking to purchase a license right away but I'm seeing some odd behavior from one of your method calls.

I'm trying to extract text from a PDF from a given region, what I wasn't expecting is that the method does a Trim() and appends 2 line breaks (\r\n\r\n) even though that's not in the data. This is critical to my usage, as I'm scanning through a region to determine where gaps are in the text and this prevents me from doing that effectively as it doesn't represent the text area accurately.

For example:

Code: Select all: var searchOptions = new PdfTextExtractOptions { IsExtractAllText = true, // preserve whitespace formatting IsSimpleExtraction = false, // fake arbitrary values ExtractArea = new RectangleF(100, 200, 200, 50) }; var text = page.ExtractText(searchOptions);

Code: Select all: full line of text: "Figure 1. Minimum Pulse Width Required For Triggering Figure 2. Supply Current vs. Supply Voltage\r\n" extracted text should be: "Figure 1. Minimum Pulse Width Required For Triggering " what it actually is: "Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n"

I've also noticed that calls to

page.ExtractText(searchOptions)

if there is no text found, it returns "\r\n".

Can this be fixed? Is there another alternative?

Wed Feb 09, 2022 1:36 am

Hello,

Thanks for your inquiry!

I simulated a simple PDF file and extract text as the code you provided with the latest Spire.PDF for net version 8.1.4, but did not reproduce your issue. Here I also attached the result screenshot and the file I tested with.

res.png

Normally, the "\r\n" is appended in the last position of one line text as a line break. To help us better investigate your issue, please provide us with the PDF file you were testing. Thanks in advance.

Sincerely,
Marcia
E-iceblue support team

Wed Feb 09, 2022 10:14 pm

Attached is the PDF I'm experiencing this on. I think the difference here is in your test, you are pulling the full width of the text with the whitespace in between characters. If you adjust your width to stop somewhere in the whitespace you will probably replicate my results.

The below code can be run to reproduce the output. For each image in the PDF, it scans vertically for text above/below the image and then horizontally to get the full text width.

The code used to process the file:

Code: Select all: using Spire.Pdf; using Spire.Pdf.Exporting; using Spire.Pdf.Texts; using System; using System.Drawing; namespace ExamplePdf { public class Example { public void Extract(string filename) { using var stream = System.IO.File.OpenRead(filename); var length = (int)stream.Length; var buffer = new byte[length]; stream.Read(buffer, 0, length); using var doc = new PdfDocument(); doc.LoadFromBytes(buffer); var docInfo = doc.DocumentInformation; var pageNumber = 0; foreach (PdfPageBase page in doc.Pages) { pageNumber++; var imagesInfo = page.ImagesInfo; var searchOptions = new PdfTextExtractOptions { IsExtractAllText = true, IsSimpleExtraction = false }; for (var i = 0; i < imagesInfo.Length; i++) { var imageInfo = imagesInfo[i]; ExtractImageMetadata(i + 1, page, pageNumber, imageInfo, searchOptions); } } } private void ExtractImageMetadata(int imageId, PdfPageBase page, int pageNumber, PdfImageInfo imageInfo, PdfTextExtractOptions searchOptions) { var iteration = 0; var aboveContextText = ScanForCloseTextVertically(imageId, page, imageInfo, searchOptions, -1, ref iteration); var belowContextText = ScanForCloseTextVertically(imageId, page, imageInfo, searchOptions, 1, ref iteration); // ignore results for example } private string ScanForCloseTextVertically(int imageId, PdfPageBase page, PdfImageInfo imageInfo, PdfTextExtractOptions searchOptions, int direction, ref int iteration) { // method: search in one direction until text is found. When we find text, then search // horizontally to try and capture the full width of the line var increment = 5; var maxSearchRange = increment * 10; for (var i = increment; i < maxSearchRange; i = i + increment) { iteration++; var searchBounds = new RectangleF(imageInfo.Bounds.X, imageInfo.Bounds.Y + (i * direction), imageInfo.Bounds.Width, imageInfo.Bounds.Height); searchOptions.ExtractArea = searchBounds; var text = page.ExtractText(searchOptions); Console.WriteLine($"[{imageId}:{iteration}] {text.Replace("\r", "\\r").Replace("\n", "\\n")}"); var fixedText = text.Replace("\r\n", ""); // remove line breaks if (!string.IsNullOrEmpty(fixedText)) return ScanHorizontalForText(imageId, page, imageInfo, searchOptions, searchBounds, ref iteration); } return string.Empty; } private string ScanHorizontalForText(int imageId, PdfPageBase page, PdfImageInfo imageInfo, PdfTextExtractOptions searchOptions, RectangleF verticalSearchBounds, ref int iteration) { // scan left until whitespace is found, then scan right until whitespace or linebreaks are found var increment = 5; var maxSearchRange = increment * 30; var previousText = string.Empty; var previousLeft = 0; RectangleF searchBounds; // scan to the left for (var i = increment; i < maxSearchRange; i = i + increment) { iteration++; searchBounds = new RectangleF(imageInfo.Bounds.X - i, verticalSearchBounds.Y, imageInfo.Bounds.Width, verticalSearchBounds.Height); searchOptions.ExtractArea = searchBounds; var text = page.ExtractText(searchOptions); Console.WriteLine($"[{imageId}:{iteration}] {text.Replace("\r", "\\r").Replace("\n", "\\n")}"); if (!string.IsNullOrEmpty(text) && (text.StartsWith("\r\n") || text.StartsWith("\n") || text.StartsWith("\t") || text.StartsWith(" "))) { // use previous result, proceed to other direction if (string.IsNullOrEmpty(previousText)) previousText = text; break; } previousText = text; previousLeft = i; } // start scanning right to left until we get the end of our text block for (var i = increment; i < maxSearchRange; i = i + increment) { iteration++; searchBounds = new RectangleF(imageInfo.Bounds.X - previousLeft, verticalSearchBounds.Y, imageInfo.Bounds.Width + previousLeft + i, verticalSearchBounds.Height); searchOptions.ExtractArea = searchBounds; var text = page.ExtractText(searchOptions); Console.WriteLine($"[{imageId}:{iteration}] {text.Replace("\r", "\\r").Replace("\n", "\\n")}"); if (!string.IsNullOrEmpty(text) && (text.EndsWith("\t") || text.EndsWith(" "))) { // use previous result, we have found the bounds if (string.IsNullOrEmpty(previousText)) previousText = text; return previousText; } previousText = text; } return previousText; } } }

Here is the output from the sample above:

Code: Select all: [1:1] \r\n [1:2] \r\n [1:3] Schematic Diagram\r\n\r\n [1:4] Schematic Diagram\r\n\r\n [1:5] Schematic Diagram\r\n\r\n [1:6] Schematic Diagram\r\n\r\n [1:7] Schematic Diagram\r\n\r\n [1:8] Schematic Diagram\r\n\r\n [1:9] Schematic Diagram\r\n\r\n [1:10] Schematic Diagram\r\n\r\n [1:11] Schematic Diagram\r\n\r\n [1:12] Schematic Diagram\r\n\r\n [1:13] Schematic Diagram\r\n\r\n [1:14] Schematic Diagram\r\n\r\n [1:15] Schematic Diagram\r\n\r\n [1:16] Schematic Diagram\r\n\r\n [1:17] Schematic Diagram\r\n\r\n [1:18] Schematic Diagram\r\n\r\n [1:19] Schematic Diagram\r\n\r\n [1:20] Schematic Diagram\r\n\r\n [1:21] Schematic Diagram\r\n\r\n [1:22] Schematic Diagram\r\n\r\n [1:23] Schematic Diagram\r\n\r\n [1:24] Schematic Diagram\r\n\r\n [1:25] Schematic Diagram\r\n\r\n [1:26] Schematic Diagram\r\n\r\n [1:27] Schematic Diagram\r\n\r\n [1:28] Schematic Diagram\r\n\r\n [1:29] Schematic Diagram\r\n\r\n [1:30] Schematic Diagram\r\n\r\n [1:31] Schematic Diagram\r\n\r\n [1:32] Schematic Diagram\r\n\r\n [1:33] Schematic Diagram\r\n\r\n [1:34] \r\n [1:35] \r\n [1:36] \r\n [1:37] \r\n [1:38] \r\n [1:39] \r\n [1:40] \r\n [1:41] \r\n [1:42] \r\n [1:1] \r\n [1:2] \r\n [1:3] re 1. Minimum Pulse Width Required For\r\n\r\n [1:4] ure 1. Minimum Pulse Width Required For\r\n\r\n [1:5] gure 1. Minimum Pulse Width Required F\r\n\r\n [1:6] Figure 1. Minimum Pulse Width Required\r\n\r\n [1:7] Figure 1. Minimum Pulse Width Required\r\n\r\n [1:8] Figure 1. Minimum Pulse Width Require\r\n\r\n [1:9] Figure 1. Minimum Pulse Width Required For Tr\r\n\r\n [1:10] Figure 1. Minimum Pulse Width Required For Tri\r\n\r\n [1:11] Figure 1. Minimum Pulse Width Required For Trig\r\n\r\n [1:12] Figure 1. Minimum Pulse Width Required For Trigg\r\n\r\n [1:13] Figure 1. Minimum Pulse Width Required For Trigger\r\n\r\n [1:14] Figure 1. Minimum Pulse Width Required For Triggeri\r\n\r\n [1:15] Figure 1. Minimum Pulse Width Required For Triggerin\r\n\r\n [1:16] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:17] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:18] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:19] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:20] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:21] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:22] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:23] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:24] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:25] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:26] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:27] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:28] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:29] Figure 1. Minimum Pulse Width Required For Triggering F\r\n\r\n [1:30] Figure 1. Minimum Pulse Width Required For Triggering Fig\r\n\r\n [1:31] Figure 1. Minimum Pulse Width Required For Triggering Figu\r\n\r\n [1:32] Figure 1. Minimum Pulse Width Required For Triggering Figur\r\n\r\n [1:33] Figure 1. Minimum Pulse Width Required For Triggering Figure\r\n\r\n [1:34] Figure 1. Minimum Pulse Width Required For Triggering Figure 2\r\n\r\n [1:35] Figure 1. Minimum Pulse Width Required For Triggering Figure 2.\r\n\r\n [1:36] Figure 1. Minimum Pulse Width Required For Triggering Figure 2. S\r\n\r\n [1:37] Figure 1. Minimum Pulse Width Required For Triggering Figure 2. Su\r\n\r\n [1:38] \r\n [1:39] \r\n [1:40] \r\n [1:41] \r\n [1:42] re 3. High Output Voltage vs. Output Sour\r\n\r\n [1:43] ure 3. High Output Voltage vs. Output Sou\r\n\r\n [1:44] gure 3. High Output Voltage vs. Output So\r\n\r\n [1:45] igure 3. High Output Voltage vs. Output S\r\n\r\n [1:46] Figure 3. High Output Voltage vs. Output\r\n\r\n [1:47] Figure 3. High Output Voltage vs. Outpu\r\n\r\n [1:48] Figure 3. High Output Voltage vs. Outp\r\n\r\n [1:49] Figure 3. High Output Voltage vs. Output Sourc\r\n\r\n [1:50] Figure 3. High Output Voltage vs. Output Source\r\n\r\n [1:51] Figure 3. High Output Voltage vs. Output Source C\r\n\r\n [1:52] Figure 3. High Output Voltage vs. Output Source Cu\r\n\r\n [1:53] Figure 3. High Output Voltage vs. Output Source Cur\r\n\r\n [1:54] Figure 3. High Output Voltage vs. Output Source Curre\r\n\r\n [1:55] Figure 3. High Output Voltage vs. Output Source Curren\r\n\r\n [1:56] Figure 3. High Output Voltage vs. Output Source Current\r\n\r\n [1:57] Figure 3. High Output Voltage vs. Output Source Current\r\n\r\n [1:58] Figure 3. High Output Voltage vs. Output Source Current\r\n\r\n [1:59] Figure 3. High Output Voltage vs. Output Source Current\r\n\r\n [1:60] Figure 3. High Output Voltage vs. Output Source Current\r\n\r\n [1:61] Figure 3. High Output Voltage vs. Output Source Current\r\n\r\n [1:62] Figure 3. High Output Voltage vs. Output Source Current\r\n\r\n [1:63] Figure 3. High Output Voltage vs. Output Source Current\r\n\r\n [1:64] Figure 3. High Output Voltage vs. Output Source Current\r\n\r\n [1:65] Figure 3. High Output Voltage vs. Output Source Current F\r\n\r\n [1:66] Figure 3. High Output Voltage vs. Output Source Current Fi\r\n\r\n [1:67] Figure 3. High Output Voltage vs. Output Source Current Fig\r\n\r\n [1:68] Figure 3. High Output Voltage vs. Output Source Current Figur\r\n\r\n [1:69] Figure 3. High Output Voltage vs. Output Source Current Figure\r\n\r\n [1:70] Figure 3. High Output Voltage vs. Output Source Current Figure 4\r\n\r\n [1:71] Figure 3. High Output Voltage vs. Output Source Current Figure 4.\r\n\r\n [1:72] Figure 3. High Output Voltage vs. Output Source Current Figure 4. L\r\n\r\n [1:73] Figure 3. High Output Voltage vs. Output Source Current Figure 4. Lo\r\n\r\n [1:74] Figure 3. High Output Voltage vs. Output Source Current Figure 4. Low\r\n\r\n [1:75] Figure 3. High Output Voltage vs. Output Source Current Figure 4. Low\r\n\r\n [1:76] Figure 3. High Output Voltage vs. Output Source Current Figure 4. Low O\r\n\r\n [1:77] Figure 3. High Output Voltage vs. Output Source Current Figure 4. Low Ou\r\n\r\n [2:1] \r\n [2:2] \r\n [2:3] Figure 2. Supply Current vs. Supply Vol\r\n\r\n [2:4] Figure 2. Supply Current vs. Supply Vo\r\n\r\n [2:5] Figure 2. Supply Current vs. Supply Volta\r\n\r\n [2:6] Figure 2. Supply Current vs. Supply Voltag\r\n\r\n [2:7] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:8] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:9] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:10] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:11] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:12] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:13] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:14] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:15] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:16] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:17] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:18] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:19] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:20] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:21] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:22] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:23] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:24] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:25] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:26] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:27] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:28] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:29] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:30] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:31] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:32] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:33] Figure 2. Supply Current vs. Supply Voltage\r\n\r\n [2:34] \r\n [2:35] \r\n [2:36] \r\n [2:37] \r\n [2:38] ure 4. Low Output Voltage vs. Output Sin\r\n\r\n [2:39] gure 4. Low Output Voltage vs. Output Si\r\n\r\n [2:40] igure 4. Low Output Voltage vs. Output S\r\n\r\n [2:41] Figure 4. Low Output Voltage vs. Output\r\n\r\n

The lines of interest in the output is here:

Code: Select all: [1:24] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:25] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:26] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:27] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:28] Figure 1. Minimum Pulse Width Required For Triggering\r\n\r\n [1:29] Figure 1. Minimum Pulse Width Required For Triggering F\r\n\r\n [1:30] Figure 1. Minimum Pulse Width Required For Triggering Fig\r\n\r\n [1:31] Figure 1. Minimum Pulse Width Required For Triggering Figu\r\n\r\n

What should be output here is a bunch of trailing spaces (you can see all the lines above [1:29] have no trailing whitespace), and then when it hits a non-whitespace character it suddenly shows up in the output in line [1:29]. This indicates a Trim() is being performed on the ExtractText() result, and it's appending \r\n\r\n on each line.

Hopefully that's not too complicated to understand, but the sample will compile and run without much effort.
I should note that I'm on Windows 10 64 bit, Visual Studio 2022 Pro (17.0.4), C# and the nuget package is FreeSpire.PDF 7.8.9.

Thu Feb 10, 2022 6:35 am

Hello,

Thanks for sharing more information!

I did encounter the "\r\n" as you mentioned according to your code. After further investigate, this "\r\n" is caused by that our Spire.PDF is extracting text line by line, and we add "\r\n" to the end of each line in the extraction area so that the string before "\r\n" can be as one line. Sorry that we will not change this in our product, you can use "Console.WriteLine($"[{imageId}:{iteration}] {text.Replace("\r", "").Replace("\n", "")}");" to show the text without "\r\n". Hope you can understand.

Sincerely,
Marcia
E-iceblue support team

Thu Feb 10, 2022 8:05 pm

That's too bad, both issues (trimmed whitespace and non-existent line breaks) are a deal breaker for me.

Fri Feb 11, 2022 2:56 am

Hello,

Sorry that our Spire.PDF will not change it. Even extract text in Adobe, the result will have "/r/n" in each line, and also ignore the white space of the end of each line. We just get the same result as Adobe.

Hope you can understand, and if you encounter any issues related to our product in the future, just feel free to contact us.

Sincerely,
Marcia
E-iceblue support team

Fri Feb 11, 2022 9:56 pm

Understandable, I actually modified my algorithm to work around this issue. I may be able to utilize this library still, though it's not a fun problem to solve. I noticed some of the other libraries don't do it, so we'll see

Mon Feb 14, 2022 3:19 am

Hello,

Thanks for your feedback!

Okay, thanks for your understanding.

If you encounter any issues related to our product in the future, just feel free to contact us.

Sincerely,
Marcia
E-iceblue support team