Sorting, grouping and streaming tables¶

The core Table is a faithful port of upstream's: it lays out whatever rows you give it, in the order you give them. rich_ext::table adds the data operations upstream leaves to you, and a table for rows that keep changing:

• TableData: typed rows under column definitions, with a stable multi-column sort, grouping with per-group aggregates, and a totals row.
• StreamingTable: rows under stable keys for append/update workloads under a live display. A frame re-renders only the rows that changed.
• VirtualTable: one window of a source too large to hold, with column widths that stay put while it scrolls.
• sort and group: the same operations as functions over plain rows, for building a core Table yourself.
• PlainRows and PlainTree: data into a core Table or Tree as plain text, never parsed as markup (below).

rs-rich-data adds a SQL-shaped result set on top (see SQL result sets).

Every view here builds core tables; none has its own table renderer. So the output looks exactly like a core Table, box styles, column options, ASCII fallback and all. No feature flag is needed.

Values and columns¶

A cell is a Value: Null, Int, Float, Str (literal text, never markup) or Text (styled). The type decides how the cell sorts and aggregates. The column decides how it looks:

fn services() -> TableData {
    let latency = |value: &Value| match value.as_f64() {
        Some(ms) => Text::new(format!("{ms:.0} ms")),
        None => Text::styled("-", "dim"),
    };
    let mut data = TableData::new([
        Column::new("service"),
        Column::new("region"),
        Column::new("errors").justify(Justify::Right),
        Column::new("p99").justify(Justify::Right).format(latency),
    ]);
    data.push([
        "api".into(),
        "eu".into(),
        Value::Int(3),
        Value::Float(120.0),
    ]);
    data.push(["web".into(), "us".into(), Value::Int(0), Value::Float(80.0)]);
    data.push(["db".into(), "eu".into(), Value::Int(7), Value::Null]);
    data.push([
        "cache".into(),
        "us".into(),
        Value::Int(1),
        Value::Float(4.0),
    ]);
    data.push([
        "worker-10".into(),
        "eu".into(),
        Value::Int(3),
        Value::Float(95.0),
    ]);
    data.push([
        "worker-9".into(),
        Value::Null,
        Value::Int(0),
        Value::Float(60.0),
    ]);
    data
}

• Column::new(header) is left-justified. .justify(..) sets the justification and .options(ColumnOptions { .. }) sets any core column option (width, ratio, no_wrap, style, …).
• .format(|value| Text) controls how a column displays its values, as here for milliseconds or with rich_ext::format for byte sizes. Aggregates are displayed through the same formatter, except counts.
• Value converts from &str, String, Text, the integer types, f64 and Option<T> (None is Null). A row shorter than the columns is padded with Null, and extra cells are dropped.

Sorting¶

sort_by takes SortKeys in priority order. Later keys break ties between rows that are equal on the earlier ones:

// Region ascending, then errors descending; the null region sorts last.
let data = services().sort_by([SortKey::asc(1), SortKey::desc(2)]);
console.print(&data);

• Stable. Rows that are equal on every key keep the order they were added in.
• Empty cells last. A Null or empty string sorts after every other value in either direction, like the missing region above.
• Natural by default. Numbers compare numerically, and so do strings that are numbers ("10" after "9"). Other strings compare case-insensitively, with runs of digits compared by value, so worker-9 sorts before worker-10. Numbers sort before text. SortKey::asc(i).lexical() compares the plain strings byte by byte instead.
• Header indicators. Each sorted column's header gets ▲ or ▼ (styled table.sort_indicator). When there is more than one key, the indicator also shows the key's priority (▲1, ▼2).

// Natural order: `worker-9` before `worker-10`.
console.print(&services().sort_by([SortKey::asc(0)]));

On a console that can only render ASCII, the indicators are ^ and v, and the core table switches to an ASCII box:

// On an ASCII-only console (`Console::builder().ascii_only(true)`) the
// indicators read `^`/`v` and the core swaps in an ASCII box.
console.print(&services().sort_by([SortKey::desc(3)]));

Grouping and aggregates¶

GroupBy::new(column) splits the rows into groups by the value in that column. Each group renders as three parts:

1. A header row. It holds the group's key in the first column, styled table.group. When the grouped column is not the first column, the column's name is prefixed (region: eu). Empty cells form one group, shown as (empty).
2. The group's rows.
3. A summary row with the group's aggregates, styled table.aggregate. This row appears only when the grouping has aggregates.

Groups appear in the order of their first row. To get groups in key order, sort by the grouped column first.

let data = services()
    .sort_by([SortKey::asc(1), SortKey::asc(0)])
    .group_by(
        GroupBy::new(1)
            .aggregate(Aggregate::sum(2))
            .aggregate(Aggregate::max(3)),
    )
    .totals(
        "all",
        [Aggregate::count(0), Aggregate::sum(2), Aggregate::mean(3)],
    );
console.print(&data);

Summary and totals rows:

• Each aggregate is placed in its own column. If a column has more than one aggregate, they are joined with ,.
• The summary label (subtotal by default, set with GroupBy::label) goes in the first cell. If the first column has its own aggregate, the label is prefixed to it (all: 6).
• TableData::totals(label, aggregates) adds one final row computed over every row.

To compute groups without rendering them, call GroupBy::groups(&rows, &order). It returns each group's key, its row indices and its aggregate values.

Plain rows and a core Table¶

sort::sort_rows, sort::sorted_indices and sort::compare_rows work on any rows that are AsRef<[Value]>, such as Vec<Vec<Value>>. They let you keep building the core Table yourself:

use rich::Table;
use rich_ext::table::{sort::sort_rows, SortKey, Value};

let mut rows = vec![
    vec![Value::from("v1.10"), Value::Int(2)],
    vec![Value::from("v1.9"), Value::Int(5)],
];
sort_rows(&mut rows, &[SortKey::desc(1), SortKey::asc(0)]);
let mut table = Table::new();
table.add_column("version").add_column("n");
for row in &rows {
    table.add_row_text(row.iter().map(Value::to_text).collect());
}

Streaming tables¶

StreamingTable<K> keeps its rows under stable keys of type K, such as a job name, a path or a sequence number:

fn jobs() -> StreamingTable<&'static str> {
    let mut jobs = StreamingTable::new([
        Column::new("job"),
        Column::new("state"),
        Column::new("done")
            .justify(Justify::Right)
            .format(|v| match v {
                Value::Int(n) => Text::new(format!("{n}%")),
                _ => Text::new(""),
            }),
    ])
    .title("pipeline")
    .window(Window::Tail(4)); // log-like: keep the newest rows on screen

    // Keys are stable: an upsert of a known key updates its row in place.
    for job in ["fetch", "build", "test", "lint", "docs"] {
        jobs.upsert(job, [job.into(), "queued".into(), Value::Int(0)]);
    }
    jobs.update_cell(&"build", 1, "running");
    jobs.update_cell(&"build", 2, Value::Int(45));
    jobs.upsert("fetch", ["fetch".into(), "done".into(), Value::Int(100)]);
    jobs
}

StreamingTable implements Renderable, so you can put it anywhere a renderable goes, including a LiveCoordinator region (see Live and layout):

fn run_live(target: RenderTarget) -> Result<(), LiveError> {
    let mut jobs = jobs();
    let mut live = LiveCoordinator::new(std::io::stdout(), target.clone());
    let region = live.add(target.segments(&jobs))?;
    live.refresh()?;

    for done in [60, 80, 100] {
        jobs.update_cell(&"build", 2, Value::Int(done));
        // Only the `build` row is laid out again; the rest come from the cache.
        live.update(region.clone(), target.segments(&jobs))?;
        live.refresh()?;
    }
    jobs.update_cell(&"build", 1, "done");
    live.update(region, target.segments(&jobs))?;
    live.finish()?;

    let stats = jobs.stats();
    eprintln!(
        "{} frames, {} row renders, {} relayouts",
        stats.frames, stats.rows_rendered, stats.relayouts
    );
    Ok(())
}

use std::io::IsTerminal;

let interactive = std::io::stdout().is_terminal();
let capabilities = TargetCapabilities {
    width: 60,
    height: 12,
    color_system: Some(ColorSystem::Standard),
    interactive,
    unicode: true,
    hyperlinks: false,
    sixel: Support::Unsupported,
};
let kind = if interactive {
    TargetKind::Terminal
} else {
    TargetKind::PlainStream
};
let target = RenderTarget::new(kind, capabilities, Theme::default_theme());
run_live(target).expect("live output");

What a frame renders again¶

Each row caches three things: its formatted cells, their widths and its rendered lines. Rendering a frame works like this:

• Only rows written since the last frame are formatted again.
• If no column's width changed, only those rows are laid out again. The rest are copied from the cache.
• If a column's width did change, every visible row is laid out again. That happens when a new row widens a column, when the widest row is removed, or when the available width changes.
• Writing a value equal to the current one is not a change. Value::Text cells are the exception: they always count as changed, because their base style cannot be compared.
• Sorting only reorders cached rows. A change to the window only drops or adds rows.

stats() returns counters for this (frames, rows_prepared, rows_rendered, relayouts), so tests can assert what a frame re-rendered. The streamed output is byte for byte what to_table() (a core Table of the same rows, built from scratch) renders, apart from the window's indicator line. The test suite checks that equality after every step of a mixed workload.

How this stays exact: the core sizes a column from the widest cell in it and nothing else. Each changed row is rendered by a core table that holds the row plus one extra row made of each column's widest cell. So the row gets exactly the widths the full table would give it.

Gotchas:

• Cached lines keep the styles they were rendered with. If you render the same table on consoles with different themes, call invalidate() between them.
• Row separators (show_lines) are not supported. A frame can have a title, a caption, a box style, and show_edge, expand and border_style.
• to_data() takes a snapshot of the rows in display order as a TableData, for grouping and aggregates.

Virtualised tables¶

VirtualTable<S> renders the rows offset..offset + height of a source and fetches nothing else, so a million-row source costs one window of memory. The source implements VirtualRows:

Vec<Vec<Value>>, slices of rows, references, Box and Arc implement it, FnRows wraps a closure that makes a row, and FnCells one that makes a cell (|row, column| value), so only the cells a window shows are made:

use rich::{Console, Justify};
use rich_ext::table::{Column, FnRows, Value, VirtualTable};

// Ten million rows, computed on demand.
let rows = FnRows::new(Some(10_000_000), |i| {
    Some(vec![Value::Int(i as i64 + 1), format!("item {i}").into()])
});
let table = VirtualTable::new(
    [Column::new("id").justify(Justify::Right), Column::new("name")],
    rows,
)
.widths([8, 10])
.offset(1_000)
.height(3);
Console::new().print(&table);

┏━━━━━━━━━━┳━━━━━━━━━━━━┓
┃       id ┃ name       ┃
┡━━━━━━━━━━╇━━━━━━━━━━━━┩
│     1001 │ item 1000  │
│     1002 │ item 1001  │
│     1003 │ item 1002  │
└──────────┴────────────┘
rows 1,001–1,003 of 10,000,000

Column widths¶

Widths don't follow the window, so columns never jump while you scroll. Each column's width comes from the first of these that is set:

1. widths([…]), in column order (None skips a column).
2. The column's own ColumnOptions::width.
3. A sample: the header and the first sample(n) rows (default 100, at most 10,000), capped at max_column_width (default 40). The sample is taken once and cached; resample() takes it again.

Cells don't wrap, so a row is one line and longer text ends in …. wrap(true) wraps instead. fit_window(true) also widens sampled columns to fit the rows shown, for a window rendered once rather than scrolled.

The position line¶

Under the table, a table.position line says where the window is:

An offset past the end of a counted source shows the last full window. show_position(false) hides the line, and footnote(text) adds one more line below it. row_numbers(true) adds a # column, and null_marker("NULL") shows null cells as a dim italic NULL instead of empty, the way an empty string shows.

Wide tables: a window of columns¶

A table of thousands of columns gets a window across as well as down. Only the columns it shows are fetched (through rows_columns), sampled for their widths and laid out, so 2,000 columns cost what a screenful does:

use rich::Console;
use rich_ext::table::{Column, FnCells, Value, VirtualTable};

// 100,000 rows by 2,000 columns, each cell made when it is shown.
let cells = FnCells::new(100_000, 2_000, |r, c| Value::from(r * c));
let columns = (0..2_000).map(|c| Column::new(format!("c{c}")));
let table = VirtualTable::new(columns, cells)
    .height(2)
    .frozen_columns(1) // `c0` stays on the left
    .column_offset(40)
    .fit_columns(true); // as many as fit the width
Console::new().print(&table); // 80 columns wide

┏━━━━┳━━━━━━┳━━━━━━┳━━━━━━┳━━━━━━┳━━━━━━┳━━━━━━┳━━━━━━┳━━━━━━┳━━━━━━┳━━━━━━┓
┃ c0 ┃ c41  ┃ c42  ┃ c43  ┃ c44  ┃ c45  ┃ c46  ┃ c47  ┃ c48  ┃ c49  ┃ c50  ┃
┡━━━━╇━━━━━━╇━━━━━━╇━━━━━━╇━━━━━━╇━━━━━━╇━━━━━━╇━━━━━━╇━━━━━━╇━━━━━━╇━━━━━━┩
│ 0  │ 0    │ 0    │ 0    │ 0    │ 0    │ 0    │ 0    │ 0    │ 0    │ 0    │
│ 0  │ 41   │ 42   │ 43   │ 44   │ 45   │ 46   │ 47   │ 48   │ 49   │ 50   │
└────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┘
rows 1–2 of 100,000, columns 42–51 of 2,000

• The position line adds where the column window is when it leaves columns out: columns 42–51 of 2,000, counted from 1, the frozen columns aside. Page::column_position gives it on its own.
• Widths are fixed or sampled as for rows, one column at a time as it comes into view, and kept: scrolling back does not sample again. A fit samples the columns a few at a time, those that surely fit at max_column_width, so it looks at most one column past those it shows.
• Without any of these builders the table shows every column, as before.

In a viewport¶

VirtualTable is a static renderer. Interactive scrolling belongs to rs-rich-interact's viewport, which drives the window through these methods:

A viewport of lines lines holds lines - chrome_lines(…) rows:

use rich::Console;
use rich_ext::table::{Column, FnRows, Value, VirtualTable};

let rows = FnRows::new(Some(1_000), |i| Some(vec![Value::from(i)]));
let mut table = VirtualTable::new([Column::new("i")], rows);
let console = Console::builder().width(30).build();
table.set_height(12 - table.chrome_lines(&console, 30));
assert_eq!(console.render_export(&table).lines().count(), 12);

Rows from data files¶

With rs-rich-data, Rows implements VirtualRows. A forward-only RowSource (a CSV or JSON Lines reader, an Arrow batch stream) can't fetch by index, so rich_data::window::RowWindow reads it once. It keeps the window, the first rows (for the width sample) and the count:

use rich_data::window::RowWindow;

let window = RowWindow::reader()
    .offset(50_000)
    .len(40)
    .count(true) // read to the end for the total (the default)
    .read(source)?;
console.print(&window.to_virtual_table());

With count(false), reading stops one row past the window, and the position line reads of 50,041+.

SQL result sets¶

rich_data::sql::ResultSet shows a query result the way a database shell does. It renders through VirtualTable and needs no database connection: rows come from Rows, a RowWindow of any adapter, or any VirtualRows (ResultSet::from_parts(columns, schema, source)).

• Typed alignment. Each column's type comes from the schema (by name, else by position). Integers, floats and decimals are right-justified, booleans centred, and text, dates and timestamps left. A column without a type is right-justified when every sampled non-null cell is a number.
• NULL. A null cell reads NULL in table.null (dim italic). An empty string stays empty. null_marker changes the text.
• The row count. (3 rows), (1 row), or (1,001+ rows) when the source can't count, with the elapsed time if you give one: (3 rows, 12ms).
• Large results. limit (default 1,000, at most 10,000) and offset choose the window. A windowed result shows its position line above the count.

use std::time::Duration;

use rich_data::sql::ResultSet;
use rich_data::{DataType, Field, Rows, Schema, Value};

let schema = Schema::new([
    Field::new("id", DataType::Integer),
    Field::new("name", DataType::String),
    Field::new("active", DataType::Boolean),
    Field::new("balance", DataType::decimal()),
]);
let mut rows = Rows::new(["id", "name", "active", "balance"]).with_schema(schema);
rows.push([Value::Int(1), "ada".into(), "true".into(), Value::Float(12.5)]);
rows.push([Value::Int(2), "".into(), "false".into(), Value::Null]);
rows.push([Value::Int(10), Value::Null, "true".into(), Value::Float(-3.0)]);
console.print(&ResultSet::new(rows).elapsed(Duration::from_millis(12)));

┏━━━━┳━━━━━━┳━━━━━━━━┳━━━━━━━━━┓
┃ id ┃ name ┃ active ┃ balance ┃
┡━━━━╇━━━━━━╇━━━━━━━━╇━━━━━━━━━┩
│  1 │ ada  │  true  │    12.5 │
│  2 │      │ false  │    NULL │
│ 10 │ NULL │  true  │      -3 │
└────┴──────┴────────┴─────────┘
(3 rows, 12ms)

For a file, ResultSet::read(source, offset, limit) reads one pass of any RowSource. Run infer::Inferrer over CSV rows first so their columns get types.

Plain-text cells¶

The core Table::add_row reads each &str as console markup, as upstream reads a str: "[bold]x[/]" is a bold x, and a [ in a file name or a column name can restyle or hide what follows. Data you do not control goes in as a Text, which is never parsed. The PlainRows extension trait does that wrapping for you, and PlainTree does it for tree labels:

use rich::{Table, Tree};
use rich_ext::table::{PlainRows, PlainTree};

let mut table = Table::new();
table.add_plain_column("file");
table.add_plain_column_justify("size", rich::Justify::Right);
table.add_plain_row(["[draft] notes.md", "12 kB"]);
table.add_plain_rows([["a:smile:b.txt", "1 kB"], ["c.txt", "3 kB"]]);
table.plain_column_footer("[total]");

let mut tree = Tree::new("files");
tree.add_plain("[draft] notes.md");

Every cell, header and footer is shown exactly as written. A plain cell is a Text cell, so upstream's rules for those apply: a column's highlight does not highlight it, and emoji codes stay as written. For Columns, use Columns::from_cells with Cell::from(Text::new(item)).

Styles¶

These keys are listed in table::STYLES and included in extended_theme(). If a theme lacks a key, the renderers fall back to the default in this table. The names don't clash with upstream's table.header, table.footer, table.cell, table.title and table.caption. Everything reads the same without colour.

Accessibility¶

TableData, StreamingTable and VirtualTable implement a11y::AccessibleText through their core Table (for VirtualTable, the rows of the window, and its columns at the width given). The result is the same Table with N rows, columns: … summary that a core table produces.