HUP-TIS: A hybrid unicode-property tokenizer with adaptive template instruction stripping for word-limit validation of Indonesian research proposals

Abstract

Research-grant information systems enforce section-level word limits on proposals, but the methods behind these checks are rarely validated. This study audited word-limit validation in the research and community service information system (SIPP) of Universitas Lancang Kuning on 30 real proposal PDFs from one submission cycle. The original code of two production generations was executed end-to-end: one counts words with the PHP function str_word_count(), the other with a Unicode-property regular expression. Both were compared with hybrid unicode-property tokenizer with adaptive template instruction stripping (HUP-TIS), a proposed pipeline that anchors section headings with layout cues and aligns lines sequentially with the official template instructions. Segmentation quality was measured without ground truth from the provenance of word 5-grams, and each system's count error was decomposed exactly into segmentation, instruction-leakage, and tokenizer components. The main finding, a failure mode not described in the literature we reviewed, is instruction-induced boundary drift. Template instructions that name other sections attracted the heading patterns of both production systems. As a result, the text counted as Methods was Introduction content in 28 to 29 documents, and 69 to 76% of methods and expected results sections were flagged as over quota, against 0 to 6.7% under HUP-TIS. Segmentation caused 83.5 to 85.9% of the absolute count error and the tokenizer only 1.9 to 4.8%. The newer system, despite its better tokenizer, disagreed with HUP-TIS on more quota decisions than the legacy system (91 vs 57). HUP-TIS processed a PDF in a median of 97 ms, against about 650 ms for the production systems. The results are limited to one institution and one submission cycle; word-level validation against human annotation is ongoing.