This week, we share a major milestone in AAL’s journey to bridge artificial intelligence, domain expertise, and rigorous academic research.
Can AI read Hong Kong’s fire-safety code?
As artificial intelligence adoption accelerates across architecture, engineering, and construction, a critical question remains unanswered: can these models truly understand complex, local statutory regulations where mistakes carry real-world consequences?
To answer this question, AAL has pioneered a first-of-its-kind, self-funded research benchmark evaluating how large language models interpret the Hong Kong Code of Practice for Fire Safety in Buildings.
Instead of relying on marketing claims or generic industry benchmarks, our study built a domain-specific evaluation dataset comprising 500 questions across escape routes, fire-resisting construction, and fire service installations. Candidate questions were checked and verified through double-blind reviews by Hong Kong-registered architects and professional fire engineers.
The findings highlight both the potential and the limits of current models:
Strong at structured selection. On 400 multiple-choice questions, the better-performing models scored around 80–90% when the prompt named Hong Kong as the applicable jurisdiction. Grok-4 scored highest at 90%; several other proprietary models were close behind. The strongest open-source results were 79% in this test.
Percent of multiple-choice questions answered correctly, with Hong Kong specified. Each bar is a model/configuration; higher is better. Source: Table 3(a). Fluent answers can hide inaccuracies. In open-ended evaluations scored by practicing AEC professionals, models consistently received higher ratings for clarity than for factual accuracy. In other words, a model often sounds confident and articulate even when its statutory interpretation is incomplete or flawed.
Expert rubric scores for five selected model versions. These are averaged scores on the paper’s 0–100 rating rubric—not percentages of answers that were correct. Source: Table 3(d). Drafts, not sign-offs. While language models serve as excellent drafting and exploratory tools, they cannot substitute for human statutory accountability. Verified clause references and professional sign-offs remain indispensable.
Presenting at AACL-IJCNLP in Zhuhai
We are excited to announce that our research paper has been officially accepted to AACL-IJCNLP 2026, the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics.
Our team will present the paper in person next week at the conference venue in Zhuhai. Taking our findings to one of the premier natural language processing gatherings in the region places AEC domain challenges directly in front of the global AI research community.
👉 Learn more about the conference programme
Gratitude to our contributors
This milestone would not have been possible without the collective effort of our co-authors, peer reviewers, and advisors.
We want to express our deepest gratitude to our co-authors, Le Qiu, Anne Zhang, Adeline Chan, and Emmanuele Chersoni, as well as every Hong Kong-registered architect, fire engineer, and industry professional who contributed their valuable time to review questions, validate clauses, and ground this study in professional reality. This achievement belongs to the entire community that believed in the value of independent inquiry.
Expanding our research commitment
Conducting this self-funded study reinforces our conviction that the future of AEC innovation must be grounded in rigorous, empirical evidence rather than speculative hype.
Moving forward, AAL is committed to producing more data-backed research initiatives for the community. We aim to help practitioners, developers, and policymakers understand the true boundary between what AI can automate today and where professional human expertise is essential.
If you are attending AACL-IJCNLP in Zhuhai or want to collaborate on upcoming industry research, please reply directly to this newsletter.
Best regards,
Natalie Gusawir
COO
AAL INNOVATION