Run PaddleOCR-VL on Novita

What is PaddleOCR-VL

PaddleOCR-VL is a SOTA and resource-efficient model tailored for document parsing. Its core component is PaddleOCR-VL-0.9B, a compact yet powerful vision-language model (VLM) that integrates a NaViT-style dynamic resolution visual encoder with the ERNIE-4.5-0.3B language model to enable accurate element recognition. This innovative model efficiently supports 109 languages and excels in recognizing complex elements (e.g., text, tables, formulas, and charts), while maintaining minimal resource consumption. Through comprehensive evaluations on widely used public benchmarks and in-house benchmarks, PaddleOCR-VL achieves SOTA performance in both page-level document parsing and element-level recognition. It significantly outperforms existing solutions, exhibits strong competitiveness against top-tier VLMs, and delivers fast inference speeds. These strengths make it highly suitable for practical deployment in real-world scenarios.

Key Features

1. Compact yet Powerful VLM Architecture: We present a novel vision-language model that is specifically designed for resource-efficient inference, achieving outstanding performance in element recognition. By integrating a NaViT-style dynamic high-resolution visual encoder with the lightweight ERNIE-4.5-0.3B language model, we significantly enhance the model’s recognition capabilities and decoding efficiency. This integration maintains high accuracy while reducing computational demands, making it well-suited for efficient and practical document processing applications.
2. SOTA Performance on Document Parsing: PaddleOCR-VL achieves state-of-the-art performance in both page-level document parsing and element-level recognition. It significantly outperforms existing pipeline-based solutions and exhibiting strong competitiveness against leading vision-language models (VLMs) in document parsing. Moreover, it excels in recognizing complex document elements, such as text, tables, formulas, and charts, making it suitable for a wide range of challenging content types, including handwritten text and historical documents. This makes it highly versatile and suitable for a wide range of document types and scenarios.
3. Multilingual Support: PaddleOCR-VL Supports 109 languages, covering major global languages, including but not limited to Chinese, English, Japanese, Latin, and Korean, as well as languages with different scripts and structures, such as Russian (Cyrillic script), Arabic, Hindi (Devanagari script), and Thai. This broad language coverage substantially enhances the applicability of our system to multilingual and globalized document processing scenarios. You can check the details on the official website of the project.

Run PaddleOCR-VL on Novita

Step 1: Console Entry
Launch the GPU interface and select Get Started to access deployment management.
Step 2: Package Selection
Locate PaddleOCR-VL in the template repository and begin installation sequence.
Step 3: Infrastructure Setup
Configure computing parameters including memory allocation, storage requirements, and network settings. Select Deploy to implement.
Step 4: Review and Create
Double-check your configuration details and cost summary. When satisfied, click Deploy to start the creation process.
Step 5: Wait for Creation
After initiating deployment, the system will automatically redirect you to the instance management page. Your instance will be created in the background.
Step 6: Monitor Download Progress
Track the image download progress in real-time. Your instance status will change from Pulling to Running once deployment is complete. You can view detailed progress by clicking the arrow icon next to your instance name. Step 7: Verify Instance Status
Click the Logs button to view instance logs and confirm that the InvokeAI service has started properly.
Step 8: Environmental Access
Launch development space through Connect interface, then initialize Start Web Terminal.

Demo

Step 1

This is a python test case.

1import base64
2import requests
3import pathlib
4
5API_URL = "http://localhost:8080/layout-parsing"  # Service URL
6
7image_path = "./demo.jpg"
8
9# Encode local image to Base64
10with open(image_path, "rb") as file:
11    image_bytes = file.read()
12    image_data = base64.b64encode(image_bytes).decode("ascii")
13
14payload = {
15    "file": image_data,  # Base64 encoded file content or file URL
16    "fileType": 1,  # File type, 1 means image file
17}
18
19# Call the API
20response = requests.post(API_URL, json=payload)
21
22# Process the API response data
23assert response.status_code == 200
24result = response.json()["result"]
25for i, res in enumerate(result["layoutParsingResults"]):
26    print(res["prunedResult"])
27    md_dir = pathlib.Path(f"markdown_{i}")
28    md_dir.mkdir(exist_ok=True)
29    (md_dir / "doc.md").write_text(res["markdown"]["text"])
30    for img_path, img in res["markdown"]["images"].items():
31        img_path = md_dir / img_path
32        img_path.parent.mkdir(parents=True, exist_ok=True)
33        img_path.write_bytes(base64.b64decode(img))
34    print(f"Markdown document saved at {md_dir / 'doc.md'}")
35    for img_name, img in res["outputImages"].items():
36        img_path = f"{img_name}_{i}.jpg"
37        pathlib.Path(img_path).parent.mkdir(exist_ok=True)
38        with open(img_path, "wb") as f:
39            f.write(base64.b64decode(img))
40        print(f"Output image saved at {img_path}")

Step 2

Prepare the picture of that needs OCR Use the official test cases in this demo. https://github.com/PaddlePaddle/PaddleOCR/blob/main/tests/test_files/book.jpg

1curl https://raw.githubusercontent.com/PaddlePaddle/PaddleOCR/main/tests/test_files/book.jpg -o demo.jpg 

Step 3

Copy port mapping address and repace API URL in test.py file.

Step 4

Run test.py and check the result.

1{'model_settings': {'use_doc_preprocessor': False, 'use_layout_detection': True, 'use_chart_recognition': False, 'format_block_content': False}, 'parsing_res_list': [{'block_label': 'text', 'block_content': "Chances of the lottery jackpot, but it's also use combination formulas to work out the chances of the other prizes, but it all starts to get a bit fiddly so we'll move on to something else. (How to work out the other lottery chances is just one of the amazing features you'll find at: www.murderousmaths.co.uk)", 'block_bbox': [180, 0, 511, 107], 'block_id': 0, 'block_order': 1}, {'block_label': 'paragraph_title', 'block_content': 'The disappearing sum', 'block_bbox': [178, 115, 308, 132], 'block_id': 1, 'block_order': 2}, {'block_label': 'text', 'block_content': "It's Friday evening. The lovely Veronica Gumfloss has been out with the football team who have all escorted her safely back to her doorstep. It's that tender moment when each hopeful player closes his eyes and leans forward with quivering lips. Unfortunately Veronica's parents heard them clumping down the road and Veronica knows she only has time to kiss four out of the eleven of them if she's going to do it properly.", 'block_bbox': [174, 125, 505, 282], 'block_id': 2, 'block_order': 3}, {'block_label': 'image', 'block_content': '', 'block_bbox': [177, 284, 489, 468], 'block_id': 3, 'block_order': None}, {'block_label': 'vision_footnote', 'block_content': "How many choices has she got? It's  $ ^{11}C_{4} $  which is  $ \\frac{11}{4! \\times 7!} $  but for goodness sake DON'T reach for the calculator! The most brilliant thing about perms and", 'block_bbox': [163, 458, 493, 528], 'block_id': 4, 'block_order': None}, {'block_label': 'number', 'block_content': '94', 'block_bbox': [300, 545, 325, 563], 'block_id': 5, 'block_order': None}, {'block_label': 'text', 'block_content': "means that EVERYTHING ON THE BOTTOM ALWAYS CANCELS OUT! It's probably the best fun you'll ever have with a pencil so here we go...", 'block_bbox': [551, 0, 890, 76], 'block_id': 6, 'block_order': 4}, {'block_label': 'display_formula', 'block_content': ' $$ \\frac{11!}{4!\\times7!}=\\frac{11\\times10\\times9\\times8\\times7\\times6\\times5\\times4\\times3\\times2\\times1}{4\\times3\\times2\\times1\\times7\\times6\\times5\\times4\\times3\\times2\\times1} $$ ', 'block_bbox': [573, 74, 879, 124], 'block_id': 7, 'block_order': 5}, {'block_label': 'text', 'block_content': "(Before we continue, grab this book and show somebody this sum. Rub their face on it if you need to and tell them that this is the sort of thing you do for fun without a calculator these days because you're so brilliant.)", 'block_bbox': [549, 124, 887, 206], 'block_id': 8, 'block_order': 6}, {'block_label': 'text', 'block_content': "Off we go then. For starters we'll get rid of the 7!bit from top and bottom and get:", 'block_bbox': [549, 205, 886, 244], 'block_id': 9, 'block_order': 7}, {'block_label': 'display_formula', 'block_content': ' $$ \\frac{11\\times10\\times9\\times8}{4\\times3\\times2\\times1} $$ ', 'block_bbox': [676, 254, 768, 290], 'block_id': 10, 'block_order': 8}, {'block_label': 'text', 'block_content': 'Pow! That\'s already got rid of more than half the numbers. Next we\'ll see that the  $ 4 \\times 2 $  on the bottom cancels out the 8 on top (and we don\'t need that "×1" on the bottom either). We\'re left with...', 'block_bbox': [546, 300, 885, 372], 'block_id': 11, 'block_order': 9}, {'block_label': 'display_formula', 'block_content': ' $$ \\frac{11\\times10\\times9}{3} $$ ', 'block_bbox': [684, 383, 755, 416], 'block_id': 12, 'block_order': 10}, {'block_label': 'text', 'block_content': "Then the 3 on the bottom divides into the 9 on top leaving it as a 3 so all we've got now is:", 'block_bbox': [545, 429, 883, 465], 'block_id': 13, 'block_order': 11}, {'block_label': 'display_formula', 'block_content': ' $$ Veronica^{\\prime}s\\ choices=11\\times10\\times3 $$ ', 'block_bbox': [617, 476, 816, 495], 'block_id': 14, 'block_order': 12}, {'block_label': 'text', 'block_content': 'Look! No bottom.', 'block_bbox': [543, 507, 665, 529], 'block_id': 15, 'block_order': 13}, {'block_label': 'number', 'block_content': '95', 'block_bbox': [705, 554, 728, 570], 'block_id': 16, 'block_order': None}], 'layout_det_res': {'boxes': [{'cls_id': 22, 'label': 'text', 'score': 0.8623488545417786, 'coordinate': [180.37161254882812, 0, 511.5435485839844, 107.78173828125]}, {'cls_id': 17, 'label': 'paragraph_title', 'score': 0.9094090461730957, 'coordinate': [178.8297119140625, 115.10285949707031, 308.66815185546875, 132.9949188232422]}, {'cls_id': 22, 'label': 'text', 'score': 0.9703303575515747, 'coordinate': [174.90829467773438, 125.8176498413086, 505.4891052246094, 282.4457702636719]}, {'cls_id': 14, 'label': 'image', 'score': 0.9670560956001282, 'coordinate': [177.98138427734375, 284.2871398925781, 489.8233642578125, 468.6240539550781]}, {'cls_id': 24, 'label': 'vision_footnote', 'score': 0.6963039636611938, 'coordinate': [163.943603515625, 458.204833984375, 493.232666015625, 528.9574584960938]}, {'cls_id': 16, 'label': 'number', 'score': 0.8297750353813171, 'coordinate': [300.74310302734375, 545.8948364257812, 325.43939208984375, 563.2888793945312]}, {'cls_id': 22, 'label': 'text', 'score': 0.9042862057685852, 'coordinate': [551.9371948242188, 0.3308563232421875, 890.8565063476562, 76.84647369384766]}, {'cls_id': 5, 'label': 'display_formula', 'score': 0.9609742760658264, 'coordinate': [573.0150146484375, 74.92318725585938, 879.24755859375, 124.89605712890625]}, {'cls_id': 22, 'label': 'text', 'score': 0.9755771160125732, 'coordinate': [549.653076171875, 124.03761291503906, 887.7197265625, 206.6728057861328]}, {'cls_id': 22, 'label': 'text', 'score': 0.959395170211792, 'coordinate': [549.031005859375, 205.06463623046875, 886.72119140625, 244.28329467773438]}, {'cls_id': 5, 'label': 'display_formula', 'score': 0.9485629796981812, 'coordinate': [676.5474243164062, 254.51788330078125, 768.7219848632812, 290.23797607421875]}, {'cls_id': 22, 'label': 'text', 'score': 0.9793534874916077, 'coordinate': [546.6148681640625, 300.3444519042969, 885.8001708984375, 372.3039855957031]}, {'cls_id': 5, 'label': 'display_formula', 'score': 0.9466578960418701, 'coordinate': [684.8164672851562, 383.488525390625, 755.7780151367188, 416.1806640625]}, {'cls_id': 22, 'label': 'text', 'score': 0.9657009840011597, 'coordinate': [545.3870849609375, 429.8050842285156, 883.2789306640625, 465.9079284667969]}, {'cls_id': 5, 'label': 'display_formula', 'score': 0.8961516618728638, 'coordinate': [617.6002197265625, 476.957763671875, 816.8131103515625, 495.5823974609375]}, {'cls_id': 22, 'label': 'text', 'score': 0.9052585363388062, 'coordinate': [543.5634765625, 507.57012939453125, 665.855712890625, 529.0166625976562]}, {'cls_id': 16, 'label': 'number', 'score': 0.8552185893058777, 'coordinate': [705.1265869140625, 554.5432739257812, 728.734375, 570.6980590820312]}]}}
2Markdown document saved at markdown_0/doc.md
3Output image saved at layout_det_res_0.jpg
4Output image saved at layout_order_res_0.jpg