UniCoreFW

Universal Core Utility Library (Python)

UniCoreFW is a compact, batteries-included utility library that provides chainable and static helper functions across arrays (lists), objects (dicts / nested structures), functions, strings, security utilities, templates, and optional cryptography helpers.

Current version: 1.1.5

Key Capabilities

• Two usage styles
• Static: UniCoreFW.map([...], fn) / _.map([...], fn)
• Chainable: _(...).map(...).filter(...).value()
• Module-spanning API
• Functions from array, object, string, function, utils, types, security, template, and crypto are attached dynamically as static methods and as chain methods.
• Security utilities
• Input validation helpers, string sanitization, rate limiting, and audit logging.
• Template engine
• ` interpolation and simple ... ` with defensive checks.
• Optional cryptography utilities
• Fernet symmetric encryption if cryptography is installed.

Installation

From PyPI

pip install unicorefw

Optional dependency (Crypto module)

The unicorefw.crypto module requires cryptography:

pip install cryptography

Project Layout

project_root_dir/
├── unicorefw/
│   ├── __init__.py
│   ├── core.py
│   ├── array.py
│   ├── object.py
│   ├── string.py
│   ├── function.py
│   ├── utils.py
│   ├── types.py
│   ├── security.py
│   ├── template.py
│   ├── crypto.py
│   ├── supporter.py
│   └── db.py              # present (import directly as unicorefw.db)
├── examples/
│   ├── sets/
│   ├── functions.py
│   ├── task_manager.py
│   └── underscore.py
└── README.md

Quick Start

from unicorefw import _, UniCoreFW

# Chainable usage
result = (
    _([1, 2, 3, 4, 5])
    .map(lambda x: x * 2)
    .filter(lambda x: x > 5)
    .value()
)
print(result)  # [6, 8, 10]

# Static usage (either UniCoreFW.* or _.*)
print(UniCoreFW.chunk([1, 2, 3, 4, 5, 6], 2))  # [[1, 2], [3, 4], [5, 6]]
print(_.chunk([1, 2, 3, 4, 5, 6], 2))          # [[1, 2], [3, 4], [5, 6]]

How the API Works

UniCoreFW exposes two main entry points:

• UniCoreFW: the primary class providing static methods like UniCoreFW.method_name(...).
• _: a convenience factory that returns a UniCoreFWWrapper for chaining:
• _(collection).map(...).filter(...).value()
• Additionally, _.method_name(...) is available as a static shortcut.

Chaining works by applying functions across UniCoreFW’s modules in a defined order; if a function name exists in a module, it becomes both:

• a chain method on UniCoreFWWrapper, and
• a static method on UniCoreFW (without overriding earlier modules’ functions when names collide).

Core Modules (What They Provide)

Arrays (unicorefw.array)

List/array utilities: map/reduce/filter/find, chunking, flattening, set-like ops, ordering, etc.

mapped = _.map([1, 2, 3], lambda x: x * 2)                 # [2, 4, 6]
flattened = _.flatten([1, [2, [3, 4]]])                    # [1, 2, 3, 4]
chunked = _.chunk([1, 2, 3, 4, 5, 6], 2)                   # [[1,2],[3,4],[5,6]]
median = _.find_median_sorted_arrays([1, 3, 5], [2, 4, 6]) # 3.5

Objects (unicorefw.object)

Dictionary and nested-structure helpers (including safe path operations), mapping, selection, and iteration.

extended = _.extend({"a": 1}, {"b": 2}, {"c": 3})  # {"a":1,"b":2,"c":3}
print(_.has({"a": {"b": [10]}}, "a.b.0"))          # True

Strings (unicorefw.string)

String transformation and inspection utilities (case transforms, regex helpers, whitespace normalization, etc.).

from unicorefw import humanize, pascal_case, normalize_whitespace
print(humanize("hello_world_example"))          # "Hello world example"
print(pascal_case("hello world"))               # "HelloWorld"
print(normalize_whitespace("  a   b\nc\t"))     # "a b c"

Functions (unicorefw.function)

Function helpers such as debounce, once, composition/flow utilities, partial/curry variants, etc.

from unicorefw import once, debounce

only_once = once(lambda: "called")
print(only_once())  # "called"
print(only_once())  # None

Utilities (unicorefw.utils)

General helpers: unique_id, now, memoize, compress/decompress, etc.

print(_.unique_id("req-"))      # e.g. "req-1"
print(_.compress("aaabbc"))     # "3a2b1c"
print(_.decompress("3a2b1c"))   # "aaabbc"

Types (unicorefw.types)

Type predicates and helpers like is_string, is_number, is_empty, deep equality, etc.

from unicorefw import is_string, is_empty
print(is_string("x"))  # True
print(is_empty({}))    # True

Security (unicorefw.security)

Input validation, sanitization, rate limiting, and audit logging primitives.

from unicorefw.security import RateLimiter, AuditLogger, validate_type, sanitize_string

validate_type("test", str, "param")
safe = sanitize_string("  abc  ", max_length=10, allowed_chars="a-zA-Z0-9")
print(safe)  # "abc"

with RateLimiter(max_calls=100, time_window=60):
    pass

with AuditLogger(log_file="security.jsonl") as logger:
    logger.log("LOGIN", {"user_id": "42", "result": "success"})

Audit files are UTF-8 JSON Lines, owner-only on POSIX, and refuse symbolic-link destinations where the operating system supports O_NOFOLLOW. Do not put credentials, tokens, or other secrets in audit details.

Templates (unicorefw.template)

A small text-template processor plus an HTML text-node renderer:

from unicorefw.template import html_template, template

print(template("Hello, !", {"name": "Alice"}))  # "Hello, Alice!"
print(html_template("", {"value": "Alice"}))
# Alice

template() produces plain text and does not HTML-escape values. html_template() accepts untrusted values only in HTML text nodes; it rejects expressions inside attributes, tags, scripts, and styles. Template source must always be trusted.

Caller-controlled expansion uses fixed budgets:

from unicorefw.security import ResourceLimitError
from unicorefw.template import TemplateLimits, html_template
from unicorefw.utils import decompress

limits = TemplateLimits(
    max_tokens=200,
    max_nesting_depth=8,
    max_output_length=100_000,
)
rendered = html_template("", context, limits=limits)

try:
    text = decompress(
        encoded_text,
        max_output_length=250_000,
        max_compression_ratio=20,
    )
except ResourceLimitError:
    reject_input()

ResourceLimitError reports an exhausted runtime budget. Invalid settings raise InputValidationError, including values above the library hard ceilings.

Other process-local helpers are bounded by default:

from unicorefw.function import debounce
from unicorefw.object import set_
from unicorefw.regex_policy import RegexLimits
from unicorefw.string import regex_test
from unicorefw.supporter import PathLimits
from unicorefw.utils import memoize

cached_lookup = memoize(
    lookup,
    max_entries=256,
    max_weight_bytes=16 * 1024 * 1024,
    ttl_seconds=300,
)
search_changed = debounce(run_search, 250, max_pending_timers=8)
set_(record, "items[9].name", "safe", limits=PathLimits(max_list_length=100))
matched = regex_test(
    user_text,
    r"[A-Za-z0-9_-]+",
    limits=RegexLimits(max_input_length=2_000),
)

The standard-library regex engine has no portable timeout. Caller-supplied patterns therefore use a conservative default policy that rejects backreferences, special groups, and nested or adjacent repetition. Use unsafe_raw_regex() only for reviewed, developer-controlled patterns; input length remains capped when the trusted wrapper is used.

Crypto (unicorefw.crypto) (optional)

Fernet symmetric encryption utilities (requires cryptography):

from unicorefw.crypto import generate_key, encrypt_string, decrypt_string

key = generate_key()
token = encrypt_string("secret", key)
print(decrypt_string(token, key))  # "secret"

Database utilities (unicorefw.db) (module present; import directly)

The repository includes a unicorefw.db module for database helpers (multi-engine, pooling, migrations, import/export). Import it explicitly:

from unicorefw.db import CacheManager, Database
db = Database(engine="sqlite", database=":memory:")
cache = CacheManager(db, max_entries=256, max_weight_bytes=16 * 1024 * 1024)

CacheManager uses monotonic TTL expiry, LRU entry and estimated-weight limits, and isolated copies of cached query results. Call cache.clear() after writes that may invalidate a cached query; cache invalidation is not automatic.

Build dynamic queries from validated identifiers and bound values. Complex SQL requires an explicit trust marker:

from unicorefw.db import QueryBuilder, unsafe_raw_sql

rows = (
    QueryBuilder(db)
    .select("id", "name")
    .from_table("users")
    .where("active = ?", True)
    .order_by("id", "DESC")
    .limit(100)
    .execute()
)

# Only reviewed, developer-controlled SQL belongs in this wrapper.
trusted_report = unsafe_raw_sql(
    "SELECT department, COUNT(*) AS total FROM users GROUP BY department"
)

Exporter source strings represent table names. Pass a reviewed query through unsafe_raw_sql(). CSV and Excel exports neutralize formula-looking text by default; spreadsheet_safe=False opts out and preserves exact strings. HTML export escapes headers and cells. Custom CSS requires unsafe_raw_css().

Importers cap bytes, rows, and columns. Excel import also caps ZIP expansion and member count. JSON defaults to 64 MiB, 100,000 rows, and 1,000 columns:

from unicorefw.db import DataImporter

inserted = DataImporter(db).from_json(
    "users.json",
    "users",
    max_bytes=8 * 1024 * 1024,
    max_rows=25_000,
    max_columns=100,
)

CSV import counts bytes while reading and rolls its transaction back when a row limit expires. Configure limits from trusted application policy, not request parameters.

SQLite backup and restore use atomic owner-only files on POSIX. Restore builds an isolated in-memory database before it changes the target.

from unicorefw.db import BackupRestore, Database

source = Database(engine="sqlite", database="app.db")
backup_path = BackupRestore(source).backup(
    "app-backup.sql", format="sql", compress=True
)
source.close()

# The safe default requires an empty target database.
target = Database(engine="sqlite", database="restored.db")
BackupRestore(target).restore(backup_path, format="sql")
target.close()

Replacing a populated target requires both flags:

target = Database(engine="sqlite", database="existing.db")
BackupRestore(target).restore(
    backup_path,
    format="sql",
    clear_existing=True,
    allow_destructive=True,
)
target.close()

Treat SQL backups as trusted executable schema artifacts. Restore denies SQLite file attachment and writable-schema pragmas, but restored triggers and views can still contain executable database logic. The default restore limits are 512 MiB after decompression and a 200:1 gzip expansion ratio. PostgreSQL, MySQL, MongoDB, and Redis backups require their native backup tools.

Changelog Notes

Older changelog entries may not include newer internal changes. Prefer GitHub releases/tags for authoritative history.

License

BSD 3-Clause License. See LICENSE.

Contributing

PRs are welcome. Please include tests where appropriate and keep changes consistent with the library’s defensive, security-oriented design.

Run the branch-coverage gate before submitting changes:

python -m pytest tests --cov=unicorefw --cov-branch \
  --cov-report=term-missing --cov-report=xml --cov-report=json \
  --cov-fail-under=76

CI rejects regressions below the current 76% ratchet. The project target is 100% statement and branch coverage; maintainers raise the ratchet after each coverage slice.