0.7.9

• Allow table model to accept optional OCR tokens

0.7.8

• Fix: include onnx as base dependency.

0.7.7

• Fix a memory leak in DonutProcessor when using large images in numpy format
• Set the right settings for beam search size > 1
• Fix a bug that in very rare cases made the last element predicted by Chipper to have a bbox = None

0.7.6

• fix a bug where invalid zoom factor lead to exceptions; now invalid zoom factors results in no scaling of the image

0.7.5

• Improved packaging

0.7.4

• Dynamic beam search size has been implemented for Chipper, the decoding process starts with a size = 1 and changes to size = 3 if repetitions appear.
• Fixed bug when PDFMiner predicts that an image text occupies the full page and removes annotations by Chipper.
• Added random seed to Chipper text generation to avoid differences between calls to Chipper.
• Allows user to use super-gradients model if they have a callback predict function, a yaml file with names field corresponding to classes and a path to the model weights

0.7.3

• Integration of Chipperv2 and additional Chipper functionality, which includes automatic detection of GPU, bounding box prediction and hierarchical representation.
• Remove control characters from the text of all layout elements

0.7.2

• Sort elements extracted by pdfminer to get consistent result from aggregate_by_block()

• Download yolox already quantized from HF
• Pins onnxruntime<1.16

0.7.0

• Remove all OCR related code expect the table OCR code

0.6.6

• Stop passing ocr_languages parameter into paddle to avoid invalid paddle language code error, this will be fixed until we have the mapping from standard language code to paddle language code.

0.6.5

• Add functionality to keep extracted image elements while merging inferred layout with extracted layout
• Fix source property for elements generated by pdfminer.
• Add 'OCR-tesseract' and 'OCR-paddle' as sources for elements generated by OCR.

0.6.4

• add a function to automatically scale table crop images based on text height so the text height is optimum for tesseract OCR task
• add the new image auto scaling parameters to config.py

What's Changed

• feat: make table transformer parameters configurable by @badGarnet in https://github.com/Unstructured-IO/unstructured-inference/pull/224
• feat: add pre commit hook by @badGarnet in https://github.com/Unstructured-IO/unstructured-inference/pull/220

Bug fixes

• fix: padded boxes are not rescaled/shifted correctly by @badGarnet in https://github.com/Unstructured-IO/unstructured-inference/pull/229

Full Changelog: https://github.com/Unstructured-IO/unstructured-inference/compare/0.6.1...0.6.3

What's Changed

• feat: add config class by @badGarnet in https://github.com/Unstructured-IO/unstructured-inference/pull/218 This change allows a user to specific inference parameters via environment variables.
• Fix/overlapping of bboxes by @benjats07 in https://github.com/Unstructured-IO/unstructured-inference/pull/201 This change makes yolox the default model for element detection and removes duplicated or near duplicated bounding boxes in the results to reduce noise in the final elements.

Full Changelog: https://github.com/Unstructured-IO/unstructured-inference/compare/0.5.31...0.6.1

0.5.31

• Add functionality to extract and save images from the page
• Add functionality to get only "true" embedded images when extracting elements from PDF pages
• Update the layout visualization script to be able to show only image elements if need
• add an evaluation metric for table comparison based on token similarity
• fix paddle unit tests where make test fails since paddle doesn't work on M1/M2 chip locally

0.5.28

• add env variable ENTIRE_PAGE_OCR to specify using paddle or tesseract on entire page OCR

• table structure detection now pads the input image by 25 pixels in all 4 directions to improve its recall

• fix a bug in table cell to html conversion where cells spanning multiple rows are not respected in the output

• remove preprocessing for OCR in table structure transformer

0.5.23

• Add functionality to bring back embedded images in PDF

0.5.22

• Add object-detection classification probabilities to LayoutElement for all currently implemented object detection models

0.5.21

• adds safe_division to replace 0 with machine epsilon for float to avoid division by 0
• apply safe_division to area overlap calculations in unstructured_inference/inference/elements.py

0.5.20

• Adds YoloX quantized model

0.5.19

• Add functionality to supplement detected layout with elements from the full page OCR
• Add functionality to annotate any layout(extracted, inferred, OCR) on a page

0.5.18

• This version adds differentiated thresholds for (optional, alternative model) detectron2_mask_rcnn.

0.5.17

• Use OMP_THREAD_LIMIT to improve tesseract performance

0.5.16

• Fix to no longer create a directory for storing processed images
• Hot-load images for annotation

• Handle an uncaught TesseractError

0.5.14

• Add TIFF test file and TIFF filetype to test_from_image_file in test_layout

0.5.13

• Fix extracted image elements being included in layout merge

0.5.12

• Fix a pdfminer error when using process_data_with_model

0.5.11

• Add warning when chipper is used with < 300 DPI
• Use None default for dpi so defaults can be properly handled upstream

0.5.10

• Implement full-page OCR

0.5.9

• Handle exceptions from Tesseract

0.5.8

• Add alternative architecture for detectron2 (but default is unchanged)
• Updates:

Library	From	To
transformers	4.29.2	4.30.2
opencv-python	4.7.0.72	4.8.0.74
ipython	8.12.2	8.14.0

• Cache named models that have been loaded

• hotfix to handle issue storing images in a new dir when the pdf has no file extension

• Update the annotate and _get_image_array methods of PageLayout to get the image from the image_path property if the image property is None.
• Add functionality to store pdf images for later use.
• Add image_metadata property to PageLayout & set page.image to None to reduce memory usage.
• Update DocumentLayout.from_file to open only one image.
• Update load_pdf to return either Image objects or Image paths.
• Warns users that Chipper is a beta model.
• Exposed control over dpi when converting PDF to an image.
• Updated detectron2 version to avoid errors related to deprecated PIL reference

• Rename large model to chipper
• Reduced memory usage when working on PDFs
• Fix issue with table processing
• Added execution providers for CUDA and TensorRT
• Warning supression for ONNX inference on empty pages.
• Updates

Library	From	To
ruff	0.0.270	0.0.276
mypy	1.3.0	1.4.1
onnxruntime	1.15.0	1.15.1

0.5.4

• Tweak to element ordering to make it more deterministic

0.5.3

• Refactor for large model

0.5.2

• Combine inferred elements with extracted elements
• Add ruff to keep code consistent with unstructured

0.5.1

• Add annotation for pages
• Store page numbers when processing PDFs
• Hotfix to handle inference of blank pages using ONNX detectron2
• Revert ordering change to investigate examples of misordering

0.5.0

• Preserve image format in PIL.Image.Image when loading
• Added ONNX version of Detectron2 and make default model
• Remove API code, we don't serve this as a standalone API any more
• Update ordering logic to account for multicolumn documents.

0.4.4

• Fixed patches not being a package.

0.4.3

• Patch pdfminer.six to fix parsing bug

0.4.2

• Output of table extraction is now stored in text_as_html property rather than text property

0.4.1

• Added the ability to pass ocr_languages to the OCR agent for users who need non-English language packs.

0.4.0

• Added logic to partition granular elements (words, characters) by proximity
• Text extraction is now delegated to text regions rather than being handled centrally
• Fixed embedded image coordinates being interpreted differently than embedded text coordinates
• Update to how dependencies are being handled
• Update detectron2 version

0.3.2

• Allow extracting tables from higher level functions

0.3.1

• Pin protobuf version to avoid errors
• Make paddleocr an extra again

0.3.0

• Fix for text block detection
• Add paddleocr dependency to setup for x86_64 machines

0.2.11

• Fixed some cases where image elements were not being OCR'd

0.2.10

• Removed control characters from tesseract output

0.2.7

• Fixed duplicated load_pdf call

0.2.5

• Add YoloX model for images and PDFs

0.2.4

• Download default model from huggingface
• Clarify error when trying to open file that doesn't exist as an image