Skip to contents

Observe a WACZ web archive

Source: R/observe_wacz.R
observe_wacz.Rd

Creates an observational data frame from a WACZ web archive.

The function extracts structural metadata from the archive, combines page-level information with WARC index metadata, and returns one observational row for each archived web page.

The resulting object represents observations only. It intentionally avoids making semantic assertions about Records, Record Parts, Instantiations, or other archival entities. Such interpretation can be added later with wacz_to_recordset_df() or downstream semantic enrichment workflows.

Usage

observe_wacz(wacz)

Arguments

wacz

Path to a .wacz archive.

Value

A tibble containing observations extracted from the archive.

The returned object carries two attributes:

  • datapackage, containing the parsed datapackage.json metadata supplied by the WACZ archive;

  • wacz, containing the normalized path to the source archive.

Typical variables include:

  • page identifiers;

  • resource locators (URLs);

  • page titles;

  • timestamps;

  • extracted text;

  • text signatures;

  • MIME types;

  • WARC digests;

  • archive offsets;

  • version counts.

Details

The function performs the following steps:

  • extracts the WACZ archive into a temporary directory;

  • reads the archive datapackage.json;

  • parses page metadata from pages/pages.jsonl;

  • parses WARC index metadata from indexes/index.cdx;

  • collapses multiple archived versions of the same resource;

  • joins page observations with archive metadata.

The resulting observations preserve the evidence contained in the archive without interpreting its archival semantics.

References

The WACZ format specification: https://specs.webrecorder.net/wacz/1.1.1/

See also

wacz_to_recordset_df()

Examples

wacz <- system.file("testdata", "fscontext_020.wacz", package = "fscontext")

observe_wacz(wacz)
#> # A tibble: 6 × 23
#>   format page_id  title has_text resource_locator   size timestamp favicon text 
#>   <chr>  <chr>    <chr> <lgl>    <chr>             <int> <chr>     <chr>   <chr>
#> 1 NA     08w4vi6… Pack… NA       https://fsconte… 1.01e6 2026-06-… https:… "Pac…
#> 2 NA     7bi5e13… Intr… NA       https://fsconte… 1.17e6 2026-06-… https:… "Int…
#> 3 NA     7yvylsx… Arti… NA       https://fsconte… 9.77e5 2026-06-… https:… "Art…
#> 4 NA     839varp… Chan… NA       https://fsconte… 9.98e5 2026-06-… https:… "Cha…
#> 5 NA     da663zi… Hand… NA       https://fsconte… 1.17e6 2026-06-… https:… "Han…
#> 6 NA     org76u2… File… NA       https://fsconte… 1.18e6 2026-06-… https:… "Fil…
#> # ℹ 14 more variables: text_length <int>, quick_sig_text <chr>, urlkey <chr>,
#> #   cdx_timestamp <chr>, digest <chr>, mime <chr>, offset <int>, length <int>,
#> #   record_digest <chr>, status <int>, warc_filename <chr>, n_versions <int>,
#> #   archive <chr>, full_path <chr>