search

package
v0.1.5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 5, 2025 License: MIT Imports: 11 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DefaultNormalizer

func DefaultNormalizer(s string) string

DefaultNormalizer lowercases and strips punctuation.

Types

type DocOpts

type DocOpts struct {
	IndexPath   string // path to save/load the index
	LoadPath    string // directory to load documents from
	LoadContent bool
	LenPreview  int
	Compressed  bool
}

type Document

type Document struct {
	Name    string `json:"name"`
	Date    string `json:"date"`
	Preview string `json:"preview"` // first N characters, using ellipsis if truncated
	Length  int    // number of words in the document
	Content string // full content, lowercase
}

func DefaultLoader

func DefaultLoader(opts DocOpts) ([]Document, error)

DefaultLoader loads documents from the filesystem using the provided options.

func NewDoc

func NewDoc(file fs.DirEntry, opts DocOpts) (Document, error)

type Index

type Index struct {
	TMap map[string]TermFreq `json:"t_map"` // term map
	// contains filtered or unexported fields
}

Index: {docs, tMap:{term: TermFreq:{idf, tfMap:{doc1: tf1, doc2: tf2, ...}}}}

func LoadIndex

func LoadIndex(loader Loader, opts DocOpts) *Index

func NewIndex

func NewIndex(loader Loader, docOpts DocOpts) *Index

NewIndex creates a new search index from the documents loaded using the provided loader function.

func (Index) DocCount

func (idx Index) DocCount() int

DocCount returns the number of documents in the index.

func (*Index) Save

func (idx *Index) Save(path string) error

Save saves the index to a file.

func (Index) Search

func (idx Index) Search(terms []string, opts SearchOpts) ([]SearchResult, error)

Search returns an ordering of the documents based on the search terms

func (Index) TermCount

func (idx Index) TermCount() int

TermCount returns the number of unique terms in the index.

func (Index) TotalWords

func (idx Index) TotalWords() int

Return the total number of words in all documents.

type Loader

type Loader func(opts DocOpts) ([]Document, error)

Loader is a function that returns documents given some options.

type MakeDoc

type MakeDoc func(file fs.DirEntry, opts DocOpts) (Document, error)

type Normalizer

type Normalizer func(text string) string

Normalizer converts a raw document string into a cleaned version before tokenization (e.g. lowercase, strip punctuation, etc.).

type SearchOpts added in v0.1.5

type SearchOpts struct {
	Limit int
}

type SearchResult

type SearchResult struct {
	*Document
	Score float64
}

type TermFreq

type TermFreq struct {
	Idf   float64            `json:"idf"`
	TfMap map[string]float64 `json:"tf_map"` // key: doc name, value: tf in doc
}

key: Document name, value: normalized tf-idf

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL