~/bend-docscommunity

normalize.bend checks

raw source on the hub · import 0xf40d06f265422706966baaec48c3e5f2/normalize.bend as Normalize

Intake normalize — strip bidi/ZWSP/BOM, fullwidth punct, dash variants, abbr periods before space (Rom. 8 → Rom 8), collapse space

1 import
import Base

Types

type FwPh source · line 31 · raw

Data

Fullwidth ：．／ → :./ Cannot nest match on computed — use phase

type NmSt source · line 95 · raw

Data

Definitions

def char1 source · line 5 · raw

@+c:Char -> String

def u32_in source · line 8 · raw

@+x:U32 -> @+lo:U32 -> @+hi:U32 -> Bool

def is_invisible_code source · line 12 · raw

@+code:U32 -> Bool

Invisible formatting: U+061C, U+200B-200D, U+200E-200F, U+202A-202E, U+2066-2069, U+FEFF

def is_invisible source · line 24 · raw

@+c:Char -> Bool

def map_fw_go source · line 37 · raw

@fuel:Nat -> @ph:FwPh -> Char

def map_fullwidth source · line 64 · raw

@+c:Char -> Char

def is_dash_code source · line 71 · raw

@+code:U32 -> Bool

Dash variants → ASCII hyphen-minus (45) ‐‑‒–—―−﹘﹣－ = 8208-8213, 8722, 65112, 65123, 65293

def map_dash_go source · line 83 · raw

@+c:Char -> @is_d:Bool -> Char

def map_dash source · line 90 · raw

@+c:Char -> Char

def normalize_go source · line 104 · raw

@fuel:Nat -> @st:NmSt -> @rest:String -> @acc:String -> @+prev_space:Bool -> String

def normalize_intake source · line 187 · raw

@+input:String -> String