Building a Diachronic Setswana Corpus, 1800 to the Present

A diachronic corpus turns the history of Setswana into evidence that can be searched, compared and interpreted. It brings texts from different periods into one framework, revealing changes in vocabulary, spelling, meaning and usage instead of presenting the language as a timeless system. This matters for Setswana-speaking communities across Botswana, South Africa and the diaspora, including families in Australia.

Building a diachronic corpus of Setswana from 1800 to present means assembling a historical series, not simply storing old words. Each record can indicate when a term entered written use, who used it, whether it belonged to formal prose or conversation, and how its sense developed. An earliest attestation still needs care: one early printing does not prove that an item was newly coined.

Historical depth is especially valuable where Setswana has interacted with Afrikaans, English and other regional languages. Corpus evidence can distinguish inherited vocabulary from loans, trace semantic borrowing and reveal when a spelling or expression became prestigious. It also supports regional comparison among Botswana, South Africa and diaspora usage, without allowing one variety to stand for all Setswana.

For an Australian audience, the project sits within a distinctive research environment. Trove in Canberra, the State Library of Victoria and PARADISEC in Melbourne, AIATSIS’s collections and community archives in Sydney offer models for large-scale description and controlled access. Australian historical lexicography, including the Australian National Dictionary, provides technical precedent without treating Setswana as a variant of Australian English.

Designing the corpus around research questions

A useful corpus begins with questions rather than a predetermined file list. Will it document the history of particular words, regional terminology, grammatical constructions, translation problems or changes associated with contact? The scope determines the sampling, and every included source needs a stable identity. Metadata should record date, place, genre, register, contributor, pagination, spelling and transcription confidence so that researchers can move from an extracted example back to its source.

Context is inseparable from interpretation. As ad valorem taxation depends on the period-specific basis of assessed value, each lexical record depends on the date, place, genre and communicative purpose of its evidence. A token in a missionary letter, a school primer or a contemporary social-media post may use the same word while participating in a different language history.

The corpus should retain both a diplomatic layer and a searchable normalised layer. The first preserves original spelling, punctuation and errors; the second groups likely variants without erasing them. Corpus design must also balance periods, genres and communities, because raw volume can exaggerate well-preserved voices. Coverage estimates and confidence flags can make that imbalance visible rather than disguising it.

Assembling sources across two centuries

Sources from the early nineteenth century are likely to be scattered across missionary records, travellers’ accounts, letters, grammars, religious primers, legal documents and newspaper notices. Some were written by people with limited access to Setswana communities, so their descriptions require contextual reading. Provenance and the history of a text’s transmission matter as much as the word printed on the page.

Digitisation changes access but not the need for appraisal. Australian repositories such as Trove can expose newspapers and books that mention southern African languages, while PARADISEC and institutional archives in Melbourne and Sydney can preserve community records, oral histories and ephemeral material. The corpus could sit beside the author’s wider lexicographic and translation work presented on T.J. Otlogetswe’s site, giving its source catalogue and public outputs a coherent scholarly home.

Modern material should represent more than elite print. Oral interviews, radio transcripts, learner materials, social media and locally published newsletters reveal registers that standard dictionaries often miss. In Australian collections, references to public holidays, school terms and events such as the Melbourne Cup can help anchor modern dates and genres, yet borrowed newspaper vocabulary must not be mistaken for everyday Setswana usage. Copyright, consent and cultural protocols apply across the entire historical series. Australian archives such as AIATSIS offer useful models for consultation and controlled access, while Setswana authorities and communities in Botswana and South Africa must guide decisions about their own materials.

Encoding change without erasing it

OCR can accelerate collection, but nineteenth-century pages often contain damaged type, unusual fonts and inconsistent orthography. Page images and OCR text should be stored beside corrected transcriptions, with corrections traceable rather than silently replacing the source. UTF-8 text, consistent punctuation and a documented file format make the collection portable, while page coordinates allow uncertain readings to be checked by later editors.

Tokenisation and lemmatisation need historical as well as computational care. A corpus layer can separate punctuation, annotate parts of speech and link spelling variants, yet normalisation should retain the original form and an explanation of each transformation. Semantic annotation should distinguish related senses with the documentary rigour highlighted in Setswana thesaurus development, where synonymy, usage labels and changing lexical relations affect the result.

Once the data are normalised, frequency per million tokens, collocation, semantic maps and change-point analysis can show which patterns are stable and which are temporary. Sampling variation and missing sources require uncertainty estimates; a rise in a written form may reflect schooling or printing rather than a change in speech. Versioned releases, a public codebook and automated checks make later analyses reproducible and keep corrections visible.

Lexicons, contact and translation in context

A diachronic corpus can supply the evidence needed for revised dictionaries, historical word histories and themed glossaries. It clarifies whether a rare sense was local, obsolete, formal or newly productive, and whether a proposed loan follows contact evidence or only resemblance. The resulting entries can distinguish first attestation, likely period of use and later semantic drift, providing more precise information to language learners and translators.

Translation adds another layer of interpretation. The process resembles symbolic interpretation in translation, where a sign carries meanings that must be read through character, setting and cultural knowledge. A historical dictionary faces a comparable task when a modern gloss obscures the social setting of an older expression. Retaining period context, regional variation and confidence labels helps readers understand why two translations may both be defensible.

There is a ready Australian readership in universities, libraries, community organisations and translation services across Sydney, Melbourne, Canberra and other cities. Dictionary publishers and public agencies also need reliable historical evidence for terminology work, cultural interpretation and migrant-language support. Serving those markets means publishing scholarly documentation alongside accessible word histories, while keeping community-sensitive material under appropriate access controls.

A sustainable workflow for corpus building

A sustainable project can begin with a limited but revealing pilot covering 1800–1900. The pilot should test source agreements, transcription rules, OCR correction, sense annotation and rights clearance before the series expands. An advisory group including Setswana lexicographers, speakers, historians, archivists and library technologists can adjudicate uncertain forms and set priorities. Publishing the pilot’s codebook and justified gaps makes those decisions testable.

Long-term preservation requires a repository plan, persistent identifiers and clear naming rules for each source and release. Raw images, transcripts, normalised files, metadata, code and changelogs should travel together, with restricted or community-governed tiers where needed. Depositing a durable package with an Australian research repository or trusted archive reduces dependence on one workstation and lets future editors rebuild earlier versions.

The practical takeaway is to begin with a documented 1800–1900 pilot, publish its source catalogue and coding rules, and expand in dated, versioned releases after review by Setswana language experts and community representatives.