Apppublic

Underground-Digital/Workflow-Engine

sourceHugging Faceupdated 2y agoView on Hugging Face

0likes

rootapicoreragcleanerunstructuredunstructured_extra_whitespace_cleaner.py

unstructured_extra_whitespace_cleaner.py13 linesDownload Raw Back to unstructured

1"""Abstract interface for document clean implementations."""2 3from core.rag.cleaner.cleaner_base import BaseCleaner4 5 6class UnstructuredNonAsciiCharsCleaner(BaseCleaner):7    def clean(self, content) -> str:8        """clean document content."""9        from unstructured.cleaners.core import clean_extra_whitespace10 11        # Returns "ITEM 1A: RISK FACTORS"12        return clean_extra_whitespace(content)13