idsbench

idsbench is a small tool for IDS (Information Delivery Specification) files from buildingSMART. It runs entirely in your browser: there is no server, and your files are not uploaded anywhere. It is one page around one document, the IDS file: open it to see whether it is valid, add a model to see whether the model meets it, and change the file right there when something has to change.

What the page does

• An IDS file arrives (opened, dropped anywhere on the page, or started with New). It is checked at once against the official IDS 1.0 schema and the rules of the buildingSMART audit tool. The stamp says Valid or Not valid. Every finding is shown at the specification it concerns, with its sentence, its line, a short code and, on demand, the lines of the file around it. Under the stamp a list says what there is to fix: "Fix" opens the specification and puts the keyboard in the field concerned; where the engine already names the answer (a class written in other letters, a missing data type of a standard property, requirements on a prohibited specification, a missing IFC version) the fix is one press, offered only after the engine has confirmed it on the changed file, and it can be undone. A name that is one or two letters away from a real one is the engine's guess ("Did you mean"): it stays in the sentence and is not made one press. A problem that is in the text of the file and not in what it says (an attribute the standard does not know) is fixed by "Write this specification again in the standard form".
• A model arrives (optional, any time). It is read in a worker and kept there while it is the model on the page, so that a run after an edit only checks again. If the IDS file is valid the model is checked by itself: there is no Run button. The stamp becomes Pass, Fail or Incomplete, each specification shows its own state and counts, and its failing elements are listed grouped by reason. Elements that fail the same requirement are one group even when each has a value of its own in the reason (20,000 walls with 20,000 wrong names read as one reason; each wall's own value is in its row). "Download failing elements" gives the complete list as CSV for the people who model, with the requirement in words, what to fix and the storey or space of each element beside why it failed: the first column, "Row about", says "Element" on the row of a failing element and "Whole specification" on the row of a specification that failed as a whole (it needs a matching element and there is none), and "Not checked" on the row of a specification whose result is withheld (it did not fail, it has no result), so a filter on that column separates the three; a model whose only rows that do not pass are Not checked has nothing to hand over, and the button is not shown (the report records them) (UTF-8 with a byte order mark, so Excel reads it; a cell that a spreadsheet would run as a formula, one that starts with =, +, - or @, gets an apostrophe in front, which is not part of the name in the model); "Download as BCF" gives the same failures as a BCF 2.1 file (.bcfzip) for coordinators who track issues in a BCF tool (see "BCF export" below); "Download report" gives the check and the run as one JSON file, which names the bytes it is about: the size and SHA-256 of the IDS text and of the model. If the IDS file is not valid, the model waits and is checked the moment the file is.
• Compare with an earlier delivery (optional, once a model is there). A coordinator who receives version 2 of a model chooses version 1 with "Compare with an earlier delivery" in the IFC box. The model on the page stays the model the verdict is about; the earlier delivery is run against the same IDS file in the same worker, after it, and kept there apart from it (the model is checked once more in a pass of its own to collect what the comparison needs, so its result on the page is never replaced, and removing the earlier delivery runs nothing), so neither pushes the other out and an edit only checks both again. One sentence says what changed ("building-v2.ifc against building-v1.ifc: 2 specifications now fail and 1 still fails."), a table names both models with their size and SHA-256, and each specification says Now passes, Now fails, Still fails (with the failing elements before and now), Still passes, or Not checked in one (nothing is compared then). Its elements are matched by GlobalId and listed by change, then by reason as in the failure list: Fixed (failed before; the specification applies to it now and it passes), No longer checked by this specification (failed before, still in the model, but the specification does not apply to it any more, so it is not known to pass), Element removed (failed before, not in the model any more), New failure (the specification applied to it before and it passed), Newly checked by this specification, failing (in the earlier model, which the specification did not apply to), New element, failing (not in the earlier model) and Still failing. To tell these apart each run also keeps the GlobalIds of every element a specification applies to. A failing element without a GlobalId, or with a GlobalId that another entity of either model also carries, cannot be matched; it is counted in a sentence of its own, never called fixed or new. "Download the comparison" gives it as CSV for the people who model (a row per specification and a row per changed element, both model names on every row; Excel-safe as the list of failing elements), and Download report then names the earlier model by name, size and SHA-256 beside the model (earlierModel) and holds the comparison. One model at a time is chosen on purpose: of two models dropped together nothing says which is the earlier one (the date of a file on disk is not the date of a delivery), so the page goes on taking the first and naming the other, as before, and the earlier delivery is always chosen by name. Without an earlier delivery the run, the page, the CSV and the report are what they were.
• Editing in place. Every specification reads as a sentence and has Edit, which opens its form in the row. IFC classes, predefined types, standard property sets, their properties, attributes and data types are chosen from pick-lists with type-ahead, built from the IFC tables the Check engine ships. A list holds what every IFC version ticked for the specification defines, and attribute names and property sets are those of the class the specification applies to, so that nothing a list offers is rejected by the engine; a plain name stands beside the technical one ("Wall (IfcWall)") and the technical form is what goes into the file. A custom name can still be typed. A new specification starts from one of five common shapes or from an empty form. It applies to the class and its subtypes in all three IFC versions ("Every wall, including wall standard case"; one control narrows it to the exact class), is optional (a model with no doors is not failed for it; the choice is in plain view), is named from what it says until a name is typed, and a shape never fills in a value that narrows the rule beyond its title. A name typed in a standard property set that the set does not define is said at the field, and names people look for by meaning ("number" for Reference) are found through a short hand-written table (src/core/meanings.ts, an aid, not the standard). The title of the IDS is suggested beside its field and never required: the official schema needs the <title> element but not a text in it, so a file without a title is written with an empty one and is valid. After a change the file is checked again and, with a model loaded, run again after a short pause.
• No files at hand? "Example" lists ready-made IDS files; its first item, "Small demo, with a model", loads a small IDS file and model that ship inside the page.
• Ready-made IDS files. "Example", next to Open and New, lists a small demo and the IDS files that ship with idsbench, each with its source, the date it was written and what it leaves out. Choosing one opens it exactly as if you had opened that file. The files are static assets in public/examples/ (the list is index.json), fetched from the page's own origin only when you open the menu or choose one, so the first load is not larger by them. Today: our own reading of the DiBK (Norway) P13 eByggesak report, the 7 hard requirements and the warnings. They are not official DiBK files, and buildingSMART's test files are not bundled.
• Show in 3D (optional, off until pressed, once a model has a result). A modeller sees where the failures are. The toggle opens a view of the model's shapes coloured by one specification, chosen in the view (the first failing one at first): the elements it fails are red, the elements it applies to that pass are green, everything else is a ghosted grey. Pressing a failing row in the list (or "Show these in 3D" on a group) turns that specification on and brings the camera to those elements; clicking an element in 3D opens its list and highlights its row when the element is one of the (at most 200 per specification) listed; an element the page does not list highlights nothing. Drag turns, shift-drag moves, the wheel comes closer. See "The 3D view" under Known limits for what it costs.

BCF export. "Download as BCF" writes a BCF 2.1 file (<model>-failing-elements.bcfzip) with one topic for each group of failing elements that the page shows (the same specification and the same reason) and one for each specification that failed as a whole. A specification that was not checked, or that passed, has no topic. The title is the requirement in words, as in the CSV's "Requirement" column (the specification's name where it cannot be put in words), followed, where the specification has more than one group, by the reason that tells the groups apart; the description says why it failed, what to fix, how many elements fail and how many are attached, and names the specification, model and IDS file; the topic type is Issue, the status Open and the label the IDS specification's name. Each topic points at its failing elements by IFC GlobalId in the Selection of one viewpoint (Components), with no camera. At most 1,000 components go in a topic: BCF readers keep a whole selection in memory and select it in the viewer in one go, and a selection of tens of thousands freezes many of them; a larger group says "Elements attached: first 1,000 of N" in its description and the CSV stays complete. An element with no GlobalId, or one that is not 22 characters of the IFC alphabet (the schema's IfcGuid type, which would make the whole file invalid), is counted in the description and not attached; a topic of such elements has no viewpoint. The page tells groups apart up to 5,000 reasons per specification, as it does for the list; failing elements of further reasons are counted in one more topic that attaches nothing. A specification that failed as a whole has a topic with no viewpoint and so no components. Why 2.1 and not 3.0: BCF 3.0's visinfo.xsd makes a camera mandatory in every viewpoint (OrthogonalCamera or PerspectiveCamera as an xs:choice with no minOccurs), so a 3.0 file would have to carry a made-up camera that sends the viewer somewhere; in 2.1 Components and both cameras are optional (minOccurs="0"), so a viewpoint can be a selection alone. The file is written with @ifc-lite/bcf (MPL-2.0) in the worker, from the groups gathered as the failures go by, not from a list held in memory. The creation author is idsbench.

Large models. The result keeps exact counts of everything and lists 200 failing elements per specification on the page; the complete list goes straight from the worker into the downloads (as pieces the browser may hold on disk; the BCF file is made in the worker from the same stream and is small whatever the number of failures), so millions of failing elements are neither sent to the page nor kept in its memory, and the CSV and the report stay complete. The model is checked one range of entities at a time (400,000 of them, cut by count and not by how large their numbers are, so that a model whose numbers are far apart is not cut into thousands of empty ranges), so that the results of millions of elements are never all in memory at once. What a tab cannot hold is refused before it is parsed, in one sentence that says what to do: a file over 1 GB by its size, and a model with more entities than a tab holds. The entities are counted over the whole file, not estimated from a part of it, and weighed: an entity with a name or an identifier of its own (an element, a relationship, a property set) counts in full, every other (geometry, mostly) for a third, which is what each was measured to cost. The limit is 9.3 million of the first kind where the browser reports 8 GB of memory or more (the most it ever reports) or reports nothing at all (Firefox, Safari), and proportionally fewer where it reports less; the sentence gives the count and says which of these it is, and does not claim that the reader's computer was measured. The reader also has a limit of about 16 million distinct names and identifiers, and says so when it meets it. The worker tells the page that it is alive from inside its long steps (counting, parsing, checking); when it has said nothing for 30 seconds, or longer while a large model is being read (0.15 s more for every megabyte), the page offers "Stop and try again"; one that the browser ends without a word is noticed at once and said so. The IDS file is checked in a worker of the Check engine's own, which cannot say that it is alive; when that check has not answered in 30 seconds (3 s more for every megabyte of the file, about ten times what it was measured to take), the page says so and offers "Check again", and a run, which checks the file first, is refused with the same sentence and "Run again". The schema validator is given memory in proportion to the file (sixteen times its size and 32 MB, at most 4 GB; a valid file of 64 MB was measured to need about 300 MB); when it runs out of memory all the same, or fails on its own, the file is not called Not valid: the page says that it could not be checked and why, with "Check again", and there is no verdict. A model that cannot be read any more (moved, deleted or saved again after it was chosen) is said as that. The report names the model by the SHA-256 of the bytes the worker parsed, taken when it parsed them, so the record is of what was checked even when the file on disk has changed since. Tab crashes that the page cannot see coming (the browser ending a tab for memory) cannot be caught by any page; the limits are set below the sizes that were measured to end one.

What writing again leaves out. A part of the file that is changed is written from what the editor holds. Before Download the page says, part by part, what that leaves out of the opened file, in words ("Writing specification 3 ("Fire rating") again will leave out: 2 comments, the attribute fixed on a list value."), and the JSON report carries the same list (writingAgainLeavesOut). It is worked out by comparing the XML of the part as the file has it with the XML written for the same content, node by node.

Exact classes. IDS matches the exact IFC class: a specification for IfcWall does not apply to an IfcWallStandardCase. Where a chosen class has subtypes in a ticked IFC version the specification says so in one line ("This applies to IfcWall only, not to IfcWallElementedCase or IfcWallStandardCase.") and offers, as an undoable action, adding the subtypes as a list. The list holds the subtypes of every ticked version, also one that only some of them have (IfcDoorStandardCase is a class of IFC4 alone, and an IFC4 model is where a list without it would pass over the doors; the Check engine accepts such a list). The one kind left out is a class that another ticked version puts under a different class (IfcZone is an IfcSystem in IFC4 and not in IFC2X3): the line names it and says why. A class that is abstract (IfcElement, IfcBuildingElement) can never be the class of an element: the line says that the specification applies to nothing as written, and the action puts the concrete subtypes in its place.

Two rules hold throughout. A result is never shown for another version of the document than the one on screen: each result is kept with the exact text and model it was made for, and is shown only while those are current. And the file you get is the file you made: a file that is opened and not changed is checked, run and downloaded as it is, byte for byte. Once something is changed, only the specification that was changed (or the info section) is written anew; everything else, comments, layout, line ends and byte order mark included, stays as the opened file has it, and a change that is typed back returns to the opened file. The record under the stamp says "with unsaved changes" while the document differs. Where a changed part held comments, the page says before Download that they are not kept.

idsbench accepts IDS 1.0 only. A file that declares any other version (0.9.7 included) is shown as Not valid with the engine's sentence, which names the version, and cannot be edited here.

Unsaved changes are kept in sessionStorage (this browser tab, gone when it closes), together with the text of the opened file they were made to, the page asks before you leave or open another file over them, and nothing is uploaded. After a reload the changes are restored and the page says so; a model is not kept and has to be chosen again. A stored draft that cannot be read is not used, and the page says that too.

An IDS file is read in the encoding it declares (UTF-8, UTF-16, ISO-8859-1 and the others a browser knows). Bytes that are not valid in that encoding are not replaced: the file is refused with the reason. A changed file is always saved as UTF-8.

The page reads IFC files in the STEP format (.ifc) for IFC2X3, IFC4 and IFC4X3. It does not read IFCX, ifcXML or zipped IFC files. A model it cannot use is refused with a sentence that says why: cut off, zipped, in an XML format or of another IFC version. A file that is not an IDS file at all is not opened, and what is on the page stays. An IDS file and a model are told apart by what they hold, so it does not matter which box a file is put in.

With a model loaded, each specification has one of four states:

State	Meaning
Pass	The model meets the specification. For a prohibited specification: none of the elements it names is in the model.
Fail	At least one element does not meet it, a required specification matches nothing, or a prohibited specification matches something.
Nothing to check	An optional specification that matches nothing in the model. The standard counts this as a pass, and so does the headline, which says how many there are.
Not checked	No verdict: the specification names no elements to apply to, uses a text pattern that is not safe to run, or allows a number of matching elements that IDS 1.0 does not have. The headline then never says that all passed.

The record under the stamp counts "Records in the file" (every entity of the file), "Physical elements" and, when there are any, "Spaces". Physical elements are the entities of IfcElement and its subtypes (walls, doors, slabs, equipment, and the other feature elements such as IfcProjectionElement and IfcSurfaceFeature), except two kinds that are not counted: the feature subtractions (IfcFeatureElementSubtraction: IfcOpeningElement, IfcOpeningStandardCase in IFC4, IfcVoidingFeature in IFC4 and IFC4X3) and IfcVirtualElement. Spaces (IfcSpace) are counted apart, and the site, the building and the storey are neither. The count comes from the IFC tables of the model's version; where they cannot be had, it is not shown.

The IFC version a specification lists (ifcVersion) does not decide whether it is checked. The official test cases treat it as metadata ("Specification version is purely metadata and does not impact pass or fail result"), so every specification is checked, and a row whose listed versions do not include the model's version says so.

How many matching elements a specification allows has three values in IDS 1.0: at least one, any number, or none (minOccurs and maxOccurs 1 and unbounded, 0 and unbounded, 0 and 0; the developer guide says other values "are currently not allowed"). Any other number ("at most 2", "at least 4") is a problem the Check verdict reports, so the file is not run. The run engine also gives no result for such a specification when it is called by itself (its row is Not checked): the library would fail "at most 2" for three matches, and IfcTester looks only at whether each value is 0 and would pass it, so there is no answer to agree with.

An element the standard does not know, inside "Applies to" or "Must have" (a misspelt one, or one of another namespace), is a problem the Check verdict reports at its line and its specification: the official schema allows no element it does not name. The file is therefore not run; the library that runs the check would skip such an element without a word and check less than the file says.

Known limits

The 3D view. It is a chunk of its own, requested only when "Show in 3D" is pressed: the first load does not contain it (the page's first-load files grew from 4,022,981 to 4,030,132 bytes, 486,797 to 490,237 gzipped, for the toggle and the colouring; the renderer chunk is 600 KB, 174 KB gzipped, and the geometry engine's WebAssembly, 10.3 MB, 3.6 MB gzipped, is requested by the worker at the same moment). It renders with @ifc-lite/renderer (MPL-2.0), which needs WebGPU: there is no WebGL fallback in the library. A browser without WebGPU gets one sentence in place of the view and nothing is downloaded; the list and the downloads do not depend on it. It was tried in Chromium only (software WebGPU in headless mode); recent Chrome and Edge have WebGPU, and for other browsers the page says what it finds and does not claim support. Nothing is requested from the network. The shapes are made by the geometry engine of the same library in the worker, in a pass of its own: the model's file is read once more (the parsed model the worker keeps holds the data of the elements, not their shapes), and the shapes are handed to the page without copying. The check is run once more, with the model the worker keeps, to give the view every failing and every applicable GlobalId (the page lists only 200 per specification); that is a few hundred milliseconds on the models measured, and is not done while a comparison of deliveries is running. Measured with a generated model of walls that are boxes (12 triangles each), software WebGPU, headless Chromium: 20,000 walls (5 MB) 3.4 s and 0.8 GB above the empty page; 100,000 walls (25 MB, 1.2 million triangles) 22 s and 1.3 GB, the check included; 150,000 walls (37 MB, 1.8 million triangles) are refused after the pass has been stopped, with the tab at 1.3 GB and nothing opened. The view refuses what it has not been shown to hold: a model over 100 MB is not read a second time (not measured beyond the size that was refused), and the pass stops as soon as the shapes pass 1,200,000 triangles; each says so in one sentence and nothing is opened. Real models have more triangles per megabyte than boxes, so the triangle count, not the file size, is what decides. Elements without a shape (types, spaces without geometry, anything a specification names that has none) cannot be drawn; they stay in the list.

Comparing two deliveries holds both models, and sends what it matches to the page. The two models of a comparison are held in the tab at once, so they are counted against one budget, the one a single model is held to: before the earlier delivery is read it is counted beside the model already held, and a pair that does not fit is refused in one sentence (each may fit alone); the result of the model is not affected. The list of failing elements goes from the worker straight into the downloads; a comparison cannot, because it matches the elements of one model against the other, so each run made for it sends the page the GlobalIds of its model, of the elements each specification applies to and of its failing elements. These are capped at 500,000 for each model: a model with more is not compared at all (comparing part of them would call elements fixed or new that are not), and the page says so and what to do instead.

Run it locally

You need Node.js 20.19 or newer.

npm install
npm run dev

Then open http://localhost:5173.

Build

npm run build

The static site is written to dist/. Serve it with any static file server, for example npm run preview (http://localhost:4173). The built site makes no network requests of its own.

How Check was measured

Check is the package in packages/check (see its README). It validates the IDS file against the official IDS 1.0 schema, then applies the rules of the buildingSMART IDS audit tool. Measured against the official tool on the IDS files in the two buildingSMART repositories: 425 of 425 IDS 1.0 files get the same valid or invalid verdict, and for the 80 the tool rejects the same set of error numbers. One more file declares IDS 0.9.7; the tool accepts it with a warning and idsbench refuses it by design. These files were used while building, so this is a regression guard, not an independent score.

• Audit tool: buildingSMART/IDS-Audit-tool, MIT licence. Its rules and IFC tables are ported to TypeScript. idsbench is not endorsed by buildingSMART.
• Schema validator: xmllint-wasm (libxml2 compiled to WebAssembly), MIT licence.

The Check engine is not part of the first load. It is fetched when the first IDS file arrives (about 456 KB over the wire, gzip, with its WebAssembly and the IFC 4 tables); the specifications are on screen before that, read by a small reader of the page's own, and the stamp says "Checking" until the engine answers.

Accessibility

scripts/a11y-check.mjs runs axe-core (@axe-core/playwright, MPL-2.0, a development dependency only: it is not in the built site) in headless Chromium over the empty page, an IDS file that is valid, one that is not valid (a generated one and an official one), a model with a Fail result with its failing elements open, the Edit form with and without a pick-list open, a refused model, "Stop and try again" and "Check again", each in the light and the dark theme (WCAG 2.0 and 2.1 A and AA, WCAG 2.2 AA and axe's best-practice rules). Serious or critical violations fail the script; lesser ones are printed. It also drives the keyboard alone: Edit, the pick-list (combobox role, aria-expanded, aria-controls, aria-activedescendant, arrows, Escape, Enter), Done, the menu that adds a specification (Escape and Cancel return the keyboard to the button that opened it), and that the verdict and the run result are written inside a polite live region.

npm run build && node scripts/a11y-check.mjs <TestCases dir>   # takes about two minutes

Known limits: axe finds about a third of what a person using a screen reader would; this was not tried with a screen reader. Only the states above are audited (not the comparison with an earlier delivery, nor a narrow screen). Pressing Delete on a specification leaves the keyboard at the top of the page.

Privacy

Files are read and checked in your browser, in Web Workers. Unsaved changes to the IDS file are kept in sessionStorage only (cleared when the tab closes); nothing else is stored. Nothing is sent to a server, there is no analytics, and the page loads no fonts or scripts from other sites.

Libraries

Library	Used for	Licence
@ifc-lite/parser	reading the IFC model	MPL-2.0
@ifc-lite/renderer, @ifc-lite/geometry	the optional 3D view: drawing the model (WebGPU) and making its shapes; loaded only when 3D is opened	MPL-2.0
@ifc-lite/bcf	writing the BCF file (it brings fflate, MIT, and jszip, MIT or GPL-3.0 at your choice, used by its reader, not by the writer)	MPL-2.0
@ifc-lite/ids	reading the IDS and checking the model against it (it cannot write IDS; the page has its own writer)	MPL-2.0
@idsbench/check (packages/check)	checking the IDS file itself, and the IFC tables behind the pick-lists; uses xmllint-wasm and @xmldom/xmldom
react, react-dom	the interface	MIT
Atkinson Hyperlegible Next, Barlow Condensed	the two typefaces, bundled as local files in src/fonts/	SIL OFL 1.1

The ifc-lite libraries are from the ifc-lite project. How the interface looks and reads is described in DESIGN.md. The build tools (Vite, TypeScript, Vitest, Playwright) are development dependencies only.

Tests

npm test                                          # the MPL-2.0 notice on every source file, the app's unit tests, then the Check package's tests
npm run lint:headers                              # only the notice: lists the source files that lack it (node scripts/check-headers.mjs --write adds it)
npm run build && npm run check:browser -- <IDS clone> [a large .ifc]   # the seven browser scripts in headless Chromium, network blocked (scripts/ASSERTIONS.md lists what they assert)
node scripts/a11y-check.mjs <TestCases dir>        # axe-core over the main states in both themes, and the keyboard journeys of the pick-list and the menus (see Accessibility)
npm run truth -- <TestCases dir>                  # every pass-/fail- pair of the buildingSMART IDS test cases through Check and Run
npx vite-node scripts/roundtrip.ts <IDS clone> <ids.xsd with local imports> <out dir>   # read, write back and compare every IDS 1.0 file
node scripts/local-xsd.mjs <official ids.xsd> <dir with xml.xsd and XMLSchema.xsd> <out>   # the schema xmllint can read without the network
npx vite-node scripts/picklists.ts [--all-properties]   # everything the pick-lists offer, fed to the Check engine
npx vite-node scripts/fixes.ts <dir> [<dir> ...]         # every one-press fix over the IDS files the engine rejects
npx vite-node scripts/handover.ts <TestCases dir>        # CSV rows (element rows and whole-specification rows, counted apart), groups and report against the run's own counts
IDS_TESTCASES=<TestCases dir> npx vitest run tests/bcf-official.test.ts   # the BCF file of every fail- pair against the BCF 2.1 schemas (downloaded to .scratch/bcf-xsd/ on first use), topic by topic (it also runs in npm test when a clone is at .scratch/IDS)
npx vite-node scripts/make-bcf.ts <file.ids> <model.ifc> <out>   # writes <out>.bcfzip and <out>.expected.json; .scratch/venv/bin/python -I scripts/bcf-client-check.py <out> reads it back with IfcOpenShell's bcf-client (python3 -m venv .scratch/venv && .scratch/venv/bin/pip install bcf-client)
npx vite-node scripts/kept-model.ts <TestCases dir>      # a run served from the kept model against the run that read it
npx vite-node scripts/lost-check.ts <dir> [<dir> ...]    # what writing each specification again leaves out, and a proof that nothing else is
npx vite-node scripts/snapshot-results.ts <TestCases dir> <out.json>   # every result of the 307 pairs, for comparing two versions of the code (node scripts/compare-results.mjs a.json b.json)
node scripts/make-large-model.mjs <walls> <out.ifc> <pass|fail|filler> [<out.ids>]   # a model of any size, for measuring
node scripts/measure-browser.mjs <model.ifc> <spec.ids>   # time, longest freeze and peak memory of one model in the real page
node scripts/make-geometry-model.mjs <walls> <out.ifc>   # walls that have a shape (a box each), for measuring the 3D view

<IDS clone> is a clone of https://github.com/buildingSMART/IDS; <TestCases dir> is its Documentation/ImplementersDocumentation/TestCases. Set CHROMIUM_PATH to use an installed headless Chromium. The inputs are used as they are; nothing is rewritten and no file is special-cased.

Measured on the one-page app, on the machine it was built on:

• Unit tests: 833 in the app (5 of them for the bundled examples: each passes Check with 0 findings; 8 the 3D view's colouring and selection on an official fail pair, tests/states.test.ts; 2 more are skipped: they need the official tool's output on disk), 70 in the Check package (1 more is skipped: it needs the official tool's output on disk).
• truth: 307 of 307 pass- / fail- pairs give the verdict named in the file name, and Check rejects none of their IDS files. In 9 the specification lists IFC2X3 and the model is IFC4; they are checked like the others and carry a note. All 27 invalid-* files are refused because Check finds problems in them.
• Check against the official audit tool: 425 of 425 IDS 1.0 files get the same valid or invalid verdict, and for the 80 it rejects the same error numbers. One file declares IDS 0.9.7 and is refused by design.
• Round trip (read, write back in full, compare): 319 files read; 319 schema-valid, 319 parse-equal, 319 equal as XML; the verdict is the same for 307 of 307 pairs.
• The file as opened: each of the 319 valid official IDS files, opened in the page and downloaded without a change, is byte for byte the file that was opened. All 760 IDS files of the two corpora that the forms can open are cut and proved for writing back; one change to one specification leaves every character outside it untouched in all of them.
• Pick-lists against the engine (scripts/picklists.ts --all-properties): 135,858 offers (every class, predefined type, attribute of a class, property set and property of a class with its data type, and data type, for each IFC version and each combination of versions): 0 rejected. Made from the first ticked version alone, as before, 10,734 of a sample of 33,279 were rejected.
• One-press fixes (scripts/fixes.ts) over the 107 files of the two corpora the engine rejects (208 findings): 18 fixes are offered and each removes its finding; 11 files are not opened in the forms at all.
• BCF export (tests/bcf-official.test.ts) over the 120 fail- pairs: every file is valid against markup.xsd, visinfo.xsd, project.xsd and version.xsd of buildingSMART/BCF-XML (release_2_1; run with xmllint-wasm, the schemas downloaded to .scratch/ because that repository states no licence and so they are not copied here); 121 topics for 121 failing groups (119 element groups, 2 specifications that failed as a whole, which have no components); the components of each topic are the GlobalIds of its group's failing elements, in order (12 topics are of elements that have no GlobalId and attach none). IfcOpenShell's bcf-client 0.9.0 opens the files and reads the same titles, types, statuses and GlobalIds (a 2,500-wall model, a mixed model of five topics, and a sample of the official pairs). Mutations that turn the tests red: the cap raised or dropped, a GlobalId lowercased, reversed or the first one dropped, the check of the GlobalId removed. 2,500 failing walls give one topic of 1,000 components ("first 1,000 of 2,500"); 1,400,000 failing walls give a 4 KB file that reaches the browser's download 0.0 s after the click with no task of the page's thread over 50 ms (the browser's own long-task measure, scripts/scale-check.mjs; the longest freeze of the page during the whole large run was 19 ms).
• Hand-over (scripts/handover.ts) over the 307 pairs, 120 of which fail: the runs count 119 failing elements and the CSV files hold 119 element rows, specification by specification; the report holds the same 119. Two specifications fail as a whole with no element to list (both are required specifications that match nothing); each has its own row in the CSV, marked "Whole specification" in the first column, and these 2 rows are counted apart from the 119.
• Where findings land, over the 66 invalid files of the corpus (139 findings): 134 at a specification, 0 at "About this IDS", 5 at the file as a whole. In the 57 of those files that open in the forms (116 findings), all 116 lead to a field of the form; none is left without a place.
• Browser: 174 assertions for the page (5 of them the Example menu: choosing the Norway example gives Valid and 7 specifications, 0 external requests; 17 the 3D view: off by default with no renderer request, the canvas drawn red, green and ghost, framing, picking, no request but local), 91 for editing and 56 for the saved file, changes under a run, the draft, a specification that is not checked, odd input and the keyboard journeys (two more are run when scripts/review-check.mjs is given a large model), 29 for large models, the kept model and which bytes its record names, a silent or ended worker, a check of the IDS file that never answers, a model that cannot be read any more and models too large to open, 21 for exact and abstract classes, a Pset_ name that is not the standard's and Download pressed again and again, and 19 for the comparison with an earlier delivery, all passed, with no request to anything but the local server. Storage holds the sessionStorage draft and nothing else. A watch on every change of the page found no moment at which a result of another version of the document was on screen; with the guard taken out of the code, 12 assertions fail.
• Size: the first load transfers 476 KB (gzip); the tabbed app transferred 434 KB. The difference is the two typefaces (30 KB) and 13 KB more script and styles. The first IDS file fetches about 456 KB more (the engine, its WebAssembly and the IFC 4 tables).
• First check after a cold load: the specifications were on screen 62 to 77 ms after the file was chosen and the verdict after 142 to 154 ms (local server, three runs).
• A 36 MB model (638,340 entities) against a four-specification IDS file: the result appeared 0.8 s after the model was chosen. With that model loaded, typing 34 characters into a form field never blocked the page for more than 21 ms, the re-run that followed included.
• 20,000 walls that all fail one specification: the result appeared 0.8 s after the files were chosen, the page listed 200 rows and said that 19,800 more were in the downloads, and the report (2.7 MB) and the CSV each held all 20,000. A 2,000-specification IDS (1.3 MB) was checked in 0.6 s with the page never frozen for more than 170 ms (0.4 s and 105 ms before the opened text was cut and proved for writing back).
• Large models (scripts/make-large-model.mjs, scripts/measure-browser.mjs; one IDS file with one specification, "walls are named Waldo"; headless Chromium on a computer with 26 GB of memory; time from choosing the model to the result, longest freeze of the page, peak memory of all the browser's processes). Before this round, from 2.8 million walls on the page froze for seconds and a model of 4.3 million failing walls ended the tab after 40 s with no message:

model	size	entities	failing	before	after
walls, all fail	13 MB	200,000	200,000	1.3 s, freeze 122 ms, 1.2 GB	1.4 s, freeze 3 ms, 1.3 GB
walls, all fail	47 MB	700,000	700,000	4.6 s, freeze 405 ms, 2.2 GB	4.5 s, freeze 2 ms, 2.3 GB
walls, all fail	95 MB	1,400,000	1,400,000	9.4 s, freeze 793 ms, 3.1 GB	9.8 s, freeze 36 ms, 3.0 GB
walls, all fail	193 MB	2,800,000	2,800,000	20.4 s, freeze 1.8 s, 5.3 GB	20.6 s, freeze 2 ms, 5.1 GB
walls, all fail	297 MB	4,300,000	4,300,000	tab ended at 40 s, no message	33.4 s, freeze 2 ms, 4.8 GB
walls, all fail	416 MB	6,000,000	6,000,000	tab ended	52.6 s, freeze 2 ms, 5.4 GB
walls, all fail	520 MB	7,500,000	7,500,000	not measured	65.5 s, freeze 2 ms, 4.9 GB
walls, all fail	624 MB	9,000,000	-	not measured	refused after 18 s in one sentence: more distinct names and identifiers than the reader can hold (its table stops at 16.7 million)
walls, all pass	43 MB	700,000	0	3.0 s, 1.7 GB	3.2 s, 1.7 GB
walls, all pass	270 MB	4,300,000	0	21.8 s, 5.4 GB	23.9 s, 4.9 GB
3 walls among other elements	54 MB	700,000	0	1.2 s, 1.3 GB	1.4 s, 1.3 GB
3 walls among other elements	703 MB	9,000,000	0	not measured	19.2 s, 5.2 GB
3 walls among other elements	781 MB	10,200,000	0	not measured	refused before it is parsed: more entities with a name of their own than a tab holds (9.3 million)
one entity in ten with a name, the rest geometry	618 MB	12,000,000	0	not measured (the count alone refused it)	10.4 s to check after reading, peak 3.2 GB above the empty page
3 walls among other elements	939 MB	12,000,000	0	not measured	with the limit taken out of a test build: the tab ended at 29 s

Where the time goes (in Node, the same code, 4.3 million walls): reading and parsing the model a third, checking the elements against the specification half (it is the library's own work, per element), shaping the result and writing the downloads the rest; none of it is on the page's thread any more. The time of a first run did not change; what changed is what happens to the result: before, every failing element was copied to the page, grouped there and held there (a freeze of 1.8 s at 2.8 million, the end of the tab at 4.3 million); now the page receives counts, 200 rows per specification and the pieces of the two downloads. The largest model that works on this machine is the largest one the page lets in: 9.3 million entities (703 MB measured), 7.5 million of them all failing (520 MB measured); it is the memory of the parsed model (about 540 bytes an entity in the whole browser) that sets the limit, not the number of failing elements. After the first run of a model, a run after an edit does not read it again: with 700,000 entities of which 3 apply, "Checked in" is 1.00 s for the first run and 0.00 s for the run after an edit. The page stays responsive in all of these (longest freeze 2 to 36 ms).

Licence

idsbench is licensed under the Mozilla Public License 2.0 (MPL-2.0), see LICENSE. You may use it in any product, open or closed; if you change idsbench's own files and distribute them, those changes are shared under the same licence.

Third-party work that idsbench builds on or bundles keeps its own licence. NOTICE.md lists it.