Benchmarking DXF Parsing Throughput in Python

To benchmark DXF parsing throughput fairly, warm the OS page cache with an untimed run, time the ezdxf.readfile() and full model-space iteration together with time.perf_counter, take the median across several runs, and measure memory separately with tracemalloc (per-file peak) and resource.getrusage (process ceiling). Report entities per second, megabytes per second, and peak resident memory — not a single wall-clock number, which conflates disk I/O, parsing, and garbage collection. This measurement is what turns the throughput claims in the Choosing ezdxf, pydwg, or ODA for Production guide into evidence you can size a pipeline and gate a CI build on.

How ezdxf Parsing Timing Breaks Down

A naive benchmark wraps ezdxf.readfile() in a timer and reports the result. That number is almost always wrong, because it silently includes first-touch disk I/O on a cold cache, and because ezdxf does some work lazily — if you never iterate the entities, you have not measured the parse your pipeline actually performs. A fair benchmark isolates the parse-and-iterate work from the I/O that happens to precede it, and reports both rather than blending them.

The diagram below contrasts a cold run, where disk I/O dominates the timed region, with a warm run, where the same file is served from the page cache and the timed region reflects parsing. The warm-run parse segment is the number you compare across code changes.

Three metrics describe throughput completely. Entities per second normalises for drawing complexity and is the most stable figure across files of different sizes. Megabytes per second normalises for raw file size and exposes I/O-bound behaviour. Peak resident memory is the ceiling that decides how many parallel workers a machine can host. Reporting all three prevents the common mistake of optimising wall-clock time on one file and regressing memory on another. For the entity model these numbers describe, the ezdxf Deep Dive is the reference.

Production-Ready Script

The harness below measures one file with a warm cache, median-of-runs timing, GC disabled inside the timed region, and both memory instruments. It scales to a directory so you can chart throughput against file size.

# ezdxf>=1.1.0 | python>=3.9
import gc
import resource
import statistics
import time
import tracemalloc
from dataclasses import dataclass, asdict
from pathlib import Path

import ezdxf


@dataclass
class BenchResult:
    path: str
    file_mb: float
    entities: int
    parse_s: float
    entities_per_s: float
    mb_per_s: float
    alloc_peak_mb: float
    proc_rss_mb: float


def _proc_rss_mb() -> float:
    # ru_maxrss is KiB on Linux and bytes on macOS. This assumes Linux.
    return resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024.0


def benchmark_file(path: Path, warmup: int = 1, runs: int = 5) -> BenchResult:
    """Measure warm-cache parse throughput and memory for one DXF file."""
    file_mb = path.stat().st_size / (1024 * 1024)

    # 1. Warm the OS page cache: measure parsing, not first-touch disk I/O.
    for _ in range(warmup):
        ezdxf.readfile(str(path))

    # 2. Timed runs. Disable GC inside the region so collection pauses
    #    do not contaminate the parse timing; re-enable immediately after.
    durations, entity_count = [], 0
    for _ in range(runs):
        gc.collect()
        gc.disable()
        start = time.perf_counter()
        doc = ezdxf.readfile(str(path))
        entity_count = sum(1 for _ in doc.modelspace())  # force the full parse
        elapsed = time.perf_counter() - start
        gc.enable()
        durations.append(elapsed)
        del doc

    parse_s = statistics.median(durations)  # median resists warm-up outliers

    # 3. Peak Python allocation for a single parse, measured in isolation.
    gc.collect()
    tracemalloc.start()
    doc = ezdxf.readfile(str(path))
    _ = sum(1 for _ in doc.modelspace())
    _, alloc_peak = tracemalloc.get_traced_memory()
    tracemalloc.stop()
    del doc

    return BenchResult(
        path=str(path),
        file_mb=round(file_mb, 3),
        entities=entity_count,
        parse_s=round(parse_s, 4),
        entities_per_s=round(entity_count / parse_s, 1) if parse_s else 0.0,
        mb_per_s=round(file_mb / parse_s, 2) if parse_s else 0.0,
        alloc_peak_mb=round(alloc_peak / (1024 * 1024), 1),
        proc_rss_mb=round(_proc_rss_mb(), 1),
    )


def benchmark_dir(directory: Path) -> list[BenchResult]:
    """Benchmark every DXF in a directory, smallest first for a clean curve."""
    files = sorted(directory.glob("*.dxf"), key=lambda p: p.stat().st_size)
    results = [benchmark_file(p) for p in files]
    for r in results:
        print(
            f"{Path(r.path).name:32} {r.file_mb:7.2f} MB  "
            f"{r.entities_per_s:12,.0f} ent/s  "
            f"{r.mb_per_s:7.2f} MB/s  peak {r.alloc_peak_mb:6.1f} MB"
        )
    return results


if __name__ == "__main__":
    import json, sys
    out = [asdict(r) for r in benchmark_dir(Path(sys.argv[1]))]
    Path("dxf_bench.json").write_text(json.dumps(out, indent=2))

Key implementation notes:

• Warm the cache first. Without the untimed warmup, the first timed run pays first-touch disk I/O and skews the median. The warmup call is discarded deliberately.
• Force iteration. sum(1 for _ in doc.modelspace()) makes ezdxf do the full parse work; timing readfile() alone under-measures because some work is deferred.
• Disable GC in the timed region only. A collection cycle firing mid-parse adds a pause unrelated to parsing. Disable it around the measurement and re-enable straight after so the process is never left with GC off.
• Median, not mean. The median across runs resists the residual warm-up outlier and transient scheduler noise better than the mean.
• Two memory instruments. tracemalloc gives per-file peak Python allocation you can reset; ru_maxrss gives the process high-water mark that sizes worker counts. They answer different questions.

Compatibility Matrix

Fallback Strategies & Pitfalls

1. OS cache warmth is the biggest confound. A benchmark run right after boot, or after processing files larger than RAM, hits cold cache and reports throughput several times lower than steady state. Always warm the cache, and when you need cold-start numbers, measure them in a clearly separate pass — never average a cold run into a warm series.

2. Garbage collection injects phantom pauses. Python’s cyclic collector can fire during a parse and add milliseconds unrelated to ezdxf. The harness disables GC inside the timed region; if you remove that, expect noisier medians. Never leave GC disabled outside the measurement in a long-running process — it leaks reference cycles.

3. I/O and parse time are different measurements. Reporting one blended number hides which stage regressed. If a change slows the pipeline, the warm parse figure tells you whether the parser or the I/O path is responsible. Keep them separate in the output.

4. ru_maxrss units and monotonicity trip people up. It is KiB on Linux but bytes on macOS, and it only ever rises. Use it for the process ceiling, not for per-file cost; for per-file peak, reset and read tracemalloc.

5. Single-file benchmarks mislead on mixed workloads. One large drawing does not predict throughput on a directory of thousands of small ones, where per-file readfile() overhead dominates. Benchmark against a corpus that mirrors production file-size distribution, smallest to largest, and read the curve rather than a single point.

6. Setting the CI regression threshold. Store the baseline entities_per_s for each reference file and fail the build when a run drops below a tolerance band — 15% is a practical starting point that absorbs runner variance without missing real regressions. Run the gate on a dedicated runner, warm the cache, and take the median of at least five runs so noise does not flap the build.

FAQ

Related Pages

• Choosing ezdxf, pydwg, or ODA for Production — the reader decision these throughput numbers inform, with the ODA conversion route for DWG inputs
• Interoperability Decision Guides — the wider framework for choosing libraries, formats, and storage targets across CAD, GIS, and BIM
• ezdxf Deep Dive — the entity traversal and memory-aware iteration patterns the harness exercises