这是用户在 2024-9-19 10:52 为 https://app.immersivetranslate.com/pdf-pro/1278e129-84c3-433c-99af-fee32bda17e5 保存的双语快照页面,由 沉浸式翻译 提供双语支持。了解如何保存?

CRAM format specification (version 3.1)
CRAM 格式规范(3.1 版)

samtools-devel@lists.sourceforge.net

4 Sep 2024 2024 年 9 月 4 日

Abstract 摘要

The master version of this document can be found at https://github.com/samtools/hts-specs. This printing is version 4127441 from that repository, last modified on the date shown above.
本文件的主版本可在 https://github.com/samtools/hts-specs 上找到。本次打印的是该版本库中的第 4127441 版,最后修改日期如上所示。

license: Apache 2.0 许可证:Apache 2.0

1 Overview 1 概述

This specification describes the CRAM 3.0 and 3.1 formats.
本规范描述了 CRAM 3.0 和 3.1 格式。

CRAM has the following major objectives:
CRAM 的主要目标如下
  1. Significantly better lossless compression than BAM
    无损压缩效果明显优于 BAM
  2. Full compatibility with BAM
    与 BAM 完全兼容
  3. Effortless transition to CRAM from using BAM files
    从使用 BAM 文件轻松过渡到 CRAM
  4. Support for controlled loss of BAM data
    支持有控制地丢失 BAM 数据
The first three objectives allow users to take immediate advantage of the CRAM format while offering a smooth transition path from using BAM files. The fourth objective supports the exploration of different lossy compression strategies and provides a framework in which to effect these choices. Please note that the CRAM format does not impose any rules about what data should or should not be preserved. Instead, CRAM supports a wide range of lossless and lossy data preservation strategies enabling users to choose which data should be preserved.
前三个目标允许用户立即利用 CRAM 格式的优势,同时提供了从使用 BAM 文件的平稳过渡途径。第四个目标支持探索不同的有损压缩策略,并提供一个框架来实现这些选择。请注意,CRAM 格式并没有对哪些数据应该或不应该保留作出任何规定。相反,CRAM 支持多种无损和有损数据保存策略,使用户能够选择保存哪些数据。

Data in CRAM is stored either as CRAM records or using one of the general purpose compressors (gzip, bzip2). CRAM records are compressed using a number of different encoding strategies. For example, bases are reference compressed by encoding base differences rather than storing the bases themselves. 1 1 ^(1){ }^{1}
CRAM 中的数据以 CRAM 记录的形式存储,或使用其中一种通用压缩器(gzip、bzip2)存储。CRAM 记录使用多种不同的编码策略进行压缩。例如,通过编码碱基差异而不是存储碱基本身来压缩碱基。 1 1 ^(1){ }^{1}

2 Data types 2 数据类型

CRAM specification uses logical data types and storage data types; logical data types are written as words (e.g. int) while physical data types are written using single letters (e.g. i). The difference between the two is that storage data types define how logical data types are stored in CRAM. Data in CRAM is stored either as bits or bytes. Writing values as bits and bytes is described in detail below.
CRAM 规范使用逻辑数据类型和存储数据类型;逻辑数据类型以单词(如 int)的形式书写,而物理数据类型则以单个字母(如 i)的形式书写。两者的区别在于,存储数据类型定义了逻辑数据类型在 CRAM 中的存储方式。CRAM 中的数据以位或字节的形式存储。下面将详细介绍以位和字节形式写入值。

2.1 Logical data types 2.1 逻辑数据类型

Byte 字节

Signed byte ( 8 bits).
有符号字节(8 位)。

Integer 整数

Signed 32-bit integer. 有符号 32 位整数。

Long 

Signed 64 -bit integer. 有符号 64 位整数。

Array 阵列

An array of any logical data type: array<type>
任何逻辑数据类型的数组: array<type>

2.2 Writing bits to a bit stream
2.2 将比特写入比特流

A bit stream consists of a sequence of 1 s and 0 s . The bits are written most significant bit first where new bits are stacked to the right and full bytes on the left are written out. In a bit stream the last byte will be incomplete if less than 8 bits have been written to it. In this case the bits in the last byte are shifted to the left.
比特流由 1 s 和 0 s 的序列组成,比特最显著位先写入,新比特向右堆叠,左侧的完整字节被写出。在比特流中,如果写入的比特少于 8 位,则最后一个字节将是不完整的。在这种情况下,最后一个字节中的位会向左移动。

Example of writing to bit stream
写入比特流的示例

Let’s consider the following example. The table below shows a sequence of write operations:
让我们来看看下面的例子。下表显示了一系列写操作:
Operation order 操作顺序 Buffer state before 之前的缓冲状态 Written bits 书面位 Buffer state after 后的缓冲状态 Issued bytes 发布字节
1 0 × 0 0 × 0 0xx00 \times 0 1 0 × 1 0 × 1 0xx10 \times 1 -
2 0 × 1 0 × 1 0xx10 \times 1 0 0 × 2 0 × 2 0xx20 \times 2 -
3 0 × 2 0 × 2 0xx20 \times 2 11 0 × B 0 × B 0xx B0 \times B -
4 0 × B 0 × B 0xx B0 \times B 00000111 0 × 7 0 × 7 0xx70 \times 7 0 × B 0 0 × B 0 0xx B00 \times B 0
Operation order Buffer state before Written bits Buffer state after Issued bytes 1 0xx0 1 0xx1 - 2 0xx1 0 0xx2 - 3 0xx2 11 0xx B - 4 0xx B 00000111 0xx7 0xx B0| Operation order | Buffer state before | Written bits | Buffer state after | Issued bytes | | :--- | :--- | :--- | :--- | :--- | | 1 | $0 \times 0$ | 1 | $0 \times 1$ | - | | 2 | $0 \times 1$ | 0 | $0 \times 2$ | - | | 3 | $0 \times 2$ | 11 | $0 \times B$ | - | | 4 | $0 \times B$ | 00000111 | $0 \times 7$ | $0 \times B 0$ |
After flushing the above bit stream the following bytes are written: 0 x B 0 0 x B 0 0xB00 x B 0 0x70. Please note that the last byte was 0 × 7 0 × 7 0xx70 \times 7 before shifting to the left and became 0 x 70 0 x 70 0x 700 x 70 after that:
刷新上述位流后,将写入以下字节: 0 x B 0 0 x B 0 0xB00 x B 0 0x70。请注意,最后一个字节在左移之前是 0 × 7 0 × 7 0xx70 \times 7 ,左移之后变成了 0 x 70 0 x 70 0x 700 x 70
> echo "obase=16; ibase=2; 00000111" | bc
7
> echo "obase=16; ibase=2; 01110000" | bc
7 0
And the whole bit sequence:
还有整个位面序列:
echo “obase=2; ibase=16; B070” | bc
echo "obase=2; ibase=16; B070" | bc| bc

1011000001110000
When reading the bits from the bit sequence it must be known that only 12 bits are meaningful and the bit stream should not be read after that.
从比特序列中读取比特时,必须知道只有 12 比特是有意义的,之后的比特流不应再读取。

Note on writing to bit stream
关于写入比特流的说明

When writing to a bit stream both the value and the number of bits in the value must be known. This is because programming languages normally operate with bytes ( 8 bits ) and to specify which bits are to be written requires a bit-holder, for example an integer, and the number of bits in it. Equally, when reading a value from a bit stream the number of bits must be known in advance. In case of prefix codes (e.g. Huffman) all possible bit combinations are either known in advance or it is possible to calculate how many bits will follow based on the first few bits. Alternatively, two codes can be combined, where the first contains the number of bits to read.
向位流中写入数值时,必须知道数值和数值中的位数。这是因为编程语言通常使用字节(8 位)进行操作,而要指定写入的位数,就需要一个比特位,例如一个整数,以及其中的比特位数。同样,从位流中读取数值时,也必须事先知道位数。如果是前缀码(如哈夫曼),所有可能的比特组合要么事先已经知道,要么可以根据前几个比特计算出后面的比特数。另外,也可以将两个编码组合在一起,其中第一个编码包含要读取的比特数。

2.3 Writing bytes to a byte stream
2.3 将字节写入字节流

The interpretation of byte stream is straightforward. CRAM uses little endianness for bytes when applicable and defines the following storage data types:
字节流的解释非常简单。CRAM 在适用情况下对字节使用小端位,并定义了以下存储数据类型:

Boolean (bool) 布尔型 (bool)

Boolean is written as 1 -byte with 0 x 0 0 x 0 0x00 x 0 being ‘false’ and 0 x 1 being ‘true’.
布尔值以 1 字节表示, 0 x 0 0 x 0 0x00 x 0 表示 "假",0 x 1 表示 "真"。

Integer (int32) 整数 (int32)

Signed 32-bit integer, written as 4 bytes in little-endian byte order.
带符号的 32 位整数,按小双位字节顺序写成 4 个字节。

Long (int64) 长 (int64)

Signed 64-bit integer, written as 8 bytes in little-endian byte order.
带符号的 64 位整数,按小双位字节顺序写成 8 个字节。

ITF-8 integer (itf8) ITF-8 整数 (itf8)

This is an alternative way to write an integer value. The idea is similar to UTF-8 encoding and therefore this encoding is called ITF-8 (Integer Transformation Format - 8 bit).
这是写入整数值的另一种方法。其思想类似于 UTF-8 编码,因此这种编码被称为 ITF-8(整数转换格式 - 8 位)。

The most significant bits of the first byte have special meaning and are called ‘prefix’. These are 0 to 4 true bits followed by a 0 . The number of 1 's denote the number of bytes to follow. To accommodate 32 bits such representation requires 5 bytes with only 4 lower bits used in the last byte 5 .
第一个字节的最重要位具有特殊意义,称为 "前缀"。它们是 0 至 4 个真实比特,后面跟一个 0。1 的个数表示后面的字节数。为了容纳 32 位,这种表示法需要 5 个字节,最后一个字节 5 只使用 4 个低位。

LTF-8 long (ltf8)

See ITF-8 for more details. The only difference between ITF-8 and LTF-8 is the number of bytes used to encode a single value. To do so 64 bits are required and this can be done with 9 byte at most with the first byte consisting of just 1 s or 0 xFF value.
详情请参见 ITF-8。ITF-8 和 LTF-8 的唯一区别在于编码单个值所需的字节数。要做到这一点,需要 64 位,而这最多可以用 9 个字节来完成,其中第一个字节仅包含 1 s 或 0 xFF 值。

Array (array<type>) 数组 (array<type>)

A variable sized array with an explicitly written dimension. Array length is written first as integer (itf8), followed by the elements of the array.
明确写入维数的可变大小数组。数组长度首先写为整数 (itf8),然后写入数组元素。

Implicit or fixed-size arrays are also used, written as type [ ] or type [4] (for example). These have no explicit dimension included in the file format and instead rely on the specification itself to document the array size.
此外,还使用隐式或固定大小的数组,例如写为类型 [ ] 或类型 [4]。这些数组在文件格式中没有明确的尺寸,而是依靠规范本身来记录数组的大小。

Encoding 编码

Encoding is a data type that specifies how data series have been compressed. Encodings are defined as encoding<type> where the type is a logical data type as opposed to a storage data type.
编码是一种数据类型,用于指定数据序列的压缩方式。编码定义为 encoding<type>,其中类型是逻辑数据类型,而不是存储数据类型。

An encoding is written as follows. The first integer (itf8) denotes the codec id and the second integer (itf8) the number of bytes in the following encoding-specific values.
编码的写法如下。第一个整数(itf8)表示编解码器 ID,第二个整数(itf8)表示以下编码特定值的字节数。

Subexponential encoding example:
次指数编码示例
Value 价值 Type 类型 Name 名称
0x7 itf8 codec id 编解码器 id
0x2 itf8 number of bytes to follow
后续字节数
0x0 itf8 offset 胶印
0x1 itf8 K parameter K 参数
Value Type Name 0x7 itf8 codec id 0x2 itf8 number of bytes to follow 0x0 itf8 offset 0x1 itf8 K parameter| Value | Type | Name | | :--- | :--- | :--- | | 0x7 | itf8 | codec id | | 0x2 | itf8 | number of bytes to follow | | 0x0 | itf8 | offset | | 0x1 | itf8 | K parameter |
The first byte " 0 × 7 0 × 7 0xx70 \times 7 " is the codec id.
第一个字节 " 0 × 7 0 × 7 0xx70 \times 7 " 是编解码器 ID。

The next byte " 0 x 2 " denotes the length of the bytes to follow (2).
下一个字节 " 0 x 2 " 表示后面的字节长度 (2)。

The subexponential encoding has 2 parameters: integer (itf8) offset and integer (itf8) K.
亚指数编码有 2 个参数:整数 (itf8) 偏移和整数 (itf8) K。

offset = 0 x 0 = 0 = 0 x 0 = 0 =0x0=0=0 \mathrm{x} 0=0 偏移量 = 0 x 0 = 0 = 0 x 0 = 0 =0x0=0=0 \mathrm{x} 0=0
K = 0 x 1 = 1 K = 0 x 1 = 1 K=0x1=1\mathrm{K}=0 \mathrm{x} 1=1
Map 地图
A map is a collection of keys and associated values. A map with N N NN keys is written as follows:
映射是键和相关值的集合。具有 N N NN 键的映射的写法如下:
size in bytes 字节大小 N key 1 关键 1 value 1 值 1 key... 关键... value ... 价值... key N 键 N value N 值 N
size in bytes N key 1 value 1 key... value ... key N value N| size in bytes | N | key 1 | value 1 | key... | value ... | key N | value N | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
Both the size in bytes and the number of keys are written as integer (itf8). Keys and values are written according to their data types and are specific to each map.
以字节为单位的大小和键的数量都以整数(itf8)形式写入。键和值根据其数据类型书写,并针对每个映射。

String 字符串

A string is represented as byte arrays using UTF-8 format. Read names, reference sequence names and tag values with type ’ Z Z ZZ ’ are stored as UTF-8.
字符串使用 UTF-8 格式表示为字节数组。类型为" Z Z ZZ "的读取名称、引用序列名称和标记值以 UTF-8 格式存储。

3 Encodings 3 编码

Encoding is a data structure that captures information about compression details of a data series that are required to uncompress it. This could be a set of constants required to initialize a specific decompression algorithm or statistical properties of a data series or, in case of data series being stored in an external block, the block content id.
编码是一种数据结构,用于捕捉解压缩数据序列所需的压缩细节信息。这可能是初始化特定解压缩算法所需的常量集或数据序列的统计属性,如果数据序列存储在外部块中,则可能是块内容 ID。
Encoding notation is defined as the keyword ‘encoding’ followed by its data type in angular brackets, for example ‘encoding<byte>’ stands for an encoding that operates on a data series of data type ‘byte’.
编码符号定义为关键字 "encoding"(编码),后面是括号中的数据类型,例如,"encoding<byte> "表示对数据类型为 "字节 "的数据系列进行操作的编码。

Encodings may have parameters of different data types, for example the EXTERNAL encoding has only one parameter, integer id of the external block. The following encodings are defined:
编码可能有不同数据类型的参数,例如 EXTERNAL 编码只有一个参数,即外部数据块的整数 ID。定义了以下编码
Codec 编解码器 ID Parameters 参数 Comment 评论
NULL 0 none  series not preserved 未保存的系列
EXTERNAL 1 int block content id int 块内容 id

块内容标识符,用于将外部数据块与数据系列关联起来
the block content identifier used to
associate external data blocks with
data series
the block content identifier used to associate external data blocks with data series| the block content identifier used to | | :--- | | associate external data blocks with | | data series |
Deprecated (GOLOMB) 已停用(GOLOMB) 2 int offset, int M Golomb coding 戈隆编码
HUFFMAN 3 array<int>, array<int> coding with int/byte values
用入/字节值编码
BYTE_ARRAY_LEN 4

编码<int> 数组长度,编码<byte> 字节
encoding<int> array length,
encoding<byte> bytes
encoding<int> array length, encoding<byte> bytes| encoding<int> array length, | | :--- | | encoding<byte> bytes |

使用数组长度对字节数组进行编码
coding of byte arrays with array
length
coding of byte arrays with array length| coding of byte arrays with array | | :--- | | length |
BYTE_ARRAY_STOP 5

byte stop, int 外部块内容 ID
byte stop, int external block
content id
byte stop, int external block content id| byte stop, int external block | | :--- | | content id |

用 stopvalue 对字节数组进行编码
coding of byte arrays with a stop
value
coding of byte arrays with a stop value| coding of byte arrays with a stop | | :--- | | value |
BETA 6 int offset, int number of bits
int 偏移,int 位数
binary coding 二进制编码
SUBEXP 7 int offset, int K int 偏移,int K subexponential coding 次指数编码
Deprecated (GOLOMB_RICE) 已停用(GOLOMB_RICE) 8 int offset, int log 2 m log 2 m log_(2)m\log _{2} \mathrm{~m}
int 偏移,int log 2 m log 2 m log_(2)m\log _{2} \mathrm{~m}
Golomb-Rice coding 戈隆-瑞斯编码
GAMMA 9 int offset int 偏移 Elias gamma coding 埃利亚斯伽马编码
Codec ID Parameters Comment NULL 0 none series not preserved EXTERNAL 1 int block content id "the block content identifier used to associate external data blocks with data series" Deprecated (GOLOMB) 2 int offset, int M Golomb coding HUFFMAN 3 array<int>, array<int> coding with int/byte values BYTE_ARRAY_LEN 4 "encoding<int> array length, encoding<byte> bytes" "coding of byte arrays with array length" BYTE_ARRAY_STOP 5 "byte stop, int external block content id" "coding of byte arrays with a stop value" BETA 6 int offset, int number of bits binary coding SUBEXP 7 int offset, int K subexponential coding Deprecated (GOLOMB_RICE) 8 int offset, int log_(2)m Golomb-Rice coding GAMMA 9 int offset Elias gamma coding| Codec | ID | Parameters | Comment | | :--- | :--- | :--- | :--- | | NULL | 0 | none | series not preserved | | EXTERNAL | 1 | int block content id | the block content identifier used to <br> associate external data blocks with <br> data series | | Deprecated (GOLOMB) | 2 | int offset, int M | Golomb coding | | HUFFMAN | 3 | array<int>, array<int> | coding with int/byte values | | BYTE_ARRAY_LEN | 4 | encoding<int> array length, <br> encoding<byte> bytes | coding of byte arrays with array <br> length | | BYTE_ARRAY_STOP | 5 | byte stop, int external block <br> content id | coding of byte arrays with a stop <br> value | | BETA | 6 | int offset, int number of bits | binary coding | | SUBEXP | 7 | int offset, int K | subexponential coding | | Deprecated (GOLOMB_RICE) | 8 | int offset, int $\log _{2} \mathrm{~m}$ | Golomb-Rice coding | | GAMMA | 9 | int offset | Elias gamma coding |
See section 13 for more detailed descriptions of all the above coding algorithms and their parameters.
有关上述所有编码算法及其参数的详细说明,请参见第 13 节。

4 Checksums 4 校验和

The checksumming is used to ensure data integrity. The following checksumming algorithms are used in CRAM.
校验和用于确保数据完整性。CRAM 中使用了以下校验和算法。

4.1 CRC32

This is a cyclic redundancy checksum 32-bit long with the polynomial 0x04C11DB7. Please refer to ITU-T V. 42 for more details. The value of the CRC32 hash function is written as an integer.
这是一个循环冗余校验和,长度为 32 位,多项式为 0x04C11DB7。详情请参阅 ITU-T V. 42。CRC32 哈希函数的值写成整数。

4.2 CRC32 sum 4.2 CRC32 和

CRC32 sum is a combination of CRC32 values by summing up all individual CRC32 values modulo 2 32 2 32 2^(32)2^{32}.
CRC32 求和是将所有单个 CRC32 值取模为 2 32 2 32 2^(32)2^{32} 求和后的 CRC32 值组合。

5 File structure 5 文件结构

The overall CRAM file structure is described in this section. Please refer to other sections of this document for more detailed information.
本节将介绍 CRAM 文件的整体结构。更多详细信息请参阅本文件的其他章节。

A CRAM file consists of a fixed length file definition, followed by a CRAM header container, then zero or more data containers, and finally a special end-of-file container.
CRAM 文件由一个固定长度的文件定义、一个 CRAM 头容器、零个或多个数据容器以及一个特殊的文件结束容器组成。
 归档定义
File
definition
File definition| File | | :---: | | definition |
 CRAM 标头容器
CRAM Header
Container
CRAM Header Container| CRAM Header | | :---: | | Container |
 数据容器
Data
Container
Data Container| Data | | :---: | | Container |
cdots\cdots
 数据容器
Data
Container
Data Container| Data | | :---: | | Container |
CRAM EOF
Container
CRAM EOF Container| CRAM EOF | | :---: | | Container |
"File definition" "CRAM Header Container" "Data Container" cdots "Data Container" "CRAM EOF Container"| File <br> definition | CRAM Header <br> Container | Data <br> Container | $\cdots$ | Data <br> Container | CRAM EOF <br> Container | | :---: | :---: | :---: | :---: | :---: | :---: |
Figure 1: A CRAM file consists of a file definition, followed by a header container, then other containers.
图 1:CRAM 文件由文件定义、文件头容器和其他容器组成。

Containers consist of one or more blocks. The first container, called the CRAM header container, is used to store a textual header as described in the SAM specification (see the section 7.1). This container may have additional padding bytes present for purposes of permitting inline rewriting of the SAM header with small changes in size. These padding bytes are undefined, but we recommend filling with nuls. The padding bytes can either be in explicit uncompressed Block structures, or as unallocated extra space where the size of the container is larger than the combined size of blocks held within it.
容器由一个或多个块组成。第一个容器称为 CRAM 标头容器,用于存储 SAM 规范中描述的文本标头(见第 7.1 节)。该容器可能会有额外的填充字节,以允许内联重写 SAM 标头,但大小变化不大。这些填充字节是未定义的,但我们建议使用 nuls 填充。填充字节可以是明确的未压缩块结构,也可以是未分配的额外空间,即容器的大小大于其中所包含块的总和。
Figure 2: The the first container holds the CRAM header text.
图 2:第一个容器包含 CRAM 标题文本。

Each container starts with a container header structure followed by one or more blocks. The first block in each container is the compression header block giving details of how to decode data in subsequent blocks. Each block starts with a block header structure followed by the block data.
每个容器都以一个容器头结构开始,然后是一个或多个块。每个容器中的第一个块是压缩头块,详细说明如何解码后续块中的数据。每个区块都以区块头结构开始,然后是区块数据。
Figure 3: Containers as a series of blocks
图 3:作为一系列区块的容器

The blocks after the compression header are organised logically into slices. One slice may contain, for example, a contiguous region of alignment data. Slices begin with a slice header block and are followed by one or more data blocks. It is these data blocks which hold the primary bulk of CRAM data. The data blocks are further subdivided into a core data block and one or more external data blocks.
压缩头之后的数据块在逻辑上被组织成片段。例如,一个片段可包含一个连续的对齐数据区域。片段以片头块开始,后面是一个或多个数据块。这些数据块是 CRAM 数据的主要部分。数据块进一步细分为一个核心数据块和一个或多个外部数据块。
Figure 4: Slices formed from a series of concatenated blocks
图 4:由一系列连接块形成的切片

6 File definition 6 文件定义

Each CRAM file starts with a fixed length (26 bytes) definition with the following fields:
每个 CRAM 文件都以固定长度(26 字节)的定义开始,其中包含以下字段:
Data type 数据类型 Name 名称 Value 价值
byte[4] 字节[4] format magic number 格式魔数 CRAM (0x43 0x52 0x41 0x4d)
unsigned byte 无符号字节 major format number 主要格式号 3 ( 0 x 3 ) 3 ( 0 x 3 ) 3(0x3)3(0 x 3)
unsigned byte 无符号字节 minor format number 次要格式号 1 (0x1)
byte[20] 字节[20] file id 文件 id CRAM file identifier (e.g. file name or SHA1 checksum)
CRAM 文件标识符(如文件名或 SHA1 校验和)。
Data type Name Value byte[4] format magic number CRAM (0x43 0x52 0x41 0x4d) unsigned byte major format number 3(0x3) unsigned byte minor format number 1 (0x1) byte[20] file id CRAM file identifier (e.g. file name or SHA1 checksum)| Data type | Name | Value | | :--- | :--- | :--- | | byte[4] | format magic number | CRAM (0x43 0x52 0x41 0x4d) | | unsigned byte | major format number | $3(0 x 3)$ | | unsigned byte | minor format number | 1 (0x1) | | byte[20] | file id | CRAM file identifier (e.g. file name or SHA1 checksum) |
Valid CRAM major.minor version numbers are as follows:
有效的 CRAM 主版本号和次版本号如下:

1.0 The original public CRAM release.
1.0 最初的公开 CRAM 版本。

2.0 The first CRAM release implemented in both Java and C; tidied up implementation vs specification differences in 1.0 .
2.0 第一个以 Java 和 C 语言实现的 CRAM 版本;整理了 1.0 版本中实现与规范之间的差异。

2.1 Gained end of file markers; compatible with 2.0.
2.1 获得文件末尾标记;与 2.0 兼容。

3.0 Additional compression methods; header and data checksums; improvements for unsorted data.
3.0 新增压缩方法;标题和数据校验和;改进未排序数据的处理。

3.1 Additional EXTERNAL compression codecs only.
3.1 仅附加外部压缩编解码器。
CRAM 3.0 and 3.1 differ only in the list of compression methods available, so tools that output CRAM 3 without using any 3.1 codecs should write the header to indicate 3.0 in order to permit maximum compatibility.
CRAM 3.0 和 3.1 的区别仅在于可用的压缩方法列表不同,因此输出 CRAM 3 而不使用任何 3.1 编解码器的工具应在标头中注明 3.0,以实现最大的兼容性。

7 Container header structure
7 集装箱头结构

The file definition is followed by one or more containers with the following header structure where the container content is stored in the ‘blocks’ field:
文件定义之后是一个或多个容器,其标题结构如下,容器内容存储在 "块 "字段中:
Data type 数据类型 Name 名称 Value 价值
int32 length 长度

该容器中所有块(标头和数据)的长度与任何填充字节的长度之和(仅限 CRAM 标头容器);等于容器的总字节长度减去该标头结构的字节长度
the sum of the lengths of all blocks in this container
(headers and data) and any padding bytes (CRAM header
container only); equal to the total byte length of the
container minus the byte length of this header structure
the sum of the lengths of all blocks in this container (headers and data) and any padding bytes (CRAM header container only); equal to the total byte length of the container minus the byte length of this header structure| the sum of the lengths of all blocks in this container | | :--- | | (headers and data) and any padding bytes (CRAM header | | container only); equal to the total byte length of the | | container minus the byte length of this header structure |
itf8 reference sequence id 参考序列 ID

该容器中的所有片段都必须有一个与该值匹配的参考序列 ID。
reference sequence identifier or
-1 for unmapped reads
-2 for multiple reference sequences.
All slices in this container must have a reference sequence
id matching this value.
reference sequence identifier or -1 for unmapped reads -2 for multiple reference sequences. All slices in this container must have a reference sequence id matching this value.| reference sequence identifier or | | :--- | | -1 for unmapped reads | | -2 for multiple reference sequences. | | All slices in this container must have a reference sequence | | id matching this value. |
itf8

起始位置
starting position on the
reference
starting position on the reference| starting position on the | | :--- | | reference |
the alignment start position
对齐起始位置
itf8 alignment span 对齐跨度 the length of the alignment
长度
itf8 number of records 记录数 number of records in the container
容器中的记录数
ltf8 record counter 记数器 1-based sequential index of records in the file/stream.
文件/数据流中记录的基于 1 的顺序索引。
ltf8 bases 基地 number of read bases 读数基数
itf8 number of blocks 块数 the total number of blocks in this container
该容器中的区块总数
array<itf8> 数组<itf8> landmarks 地标

该容器中分片的位置,作为从该容器标头末尾开始的字节偏移量,用于随机存取索引。由于第一个分片之前的块是压缩头,因此 landmarks[0] 等于压缩头的字节长度。
the locations of slices in this container as byte offsets from
the end of this container header, used for random access
indexing. For sequence data containers, the landmark
count must equal the slice count.
Since the block before the first slice is the compression
header, landmarks[0] is equal to the byte length of the
compression header.
the locations of slices in this container as byte offsets from the end of this container header, used for random access indexing. For sequence data containers, the landmark count must equal the slice count. Since the block before the first slice is the compression header, landmarks[0] is equal to the byte length of the compression header.| the locations of slices in this container as byte offsets from | | :--- | | the end of this container header, used for random access | | indexing. For sequence data containers, the landmark | | count must equal the slice count. | | Since the block before the first slice is the compression | | header, landmarks[0] is equal to the byte length of the | | compression header. |
int crc32 CRC32 hash of the all the preceding bytes in the container.
容器中前面所有字节的 CRC32 哈希值。
byte[ 字节 blocks 大厦 The blocks contained within the container.
容器中包含的区块。
Data type Name Value int32 length "the sum of the lengths of all blocks in this container (headers and data) and any padding bytes (CRAM header container only); equal to the total byte length of the container minus the byte length of this header structure" itf8 reference sequence id "reference sequence identifier or -1 for unmapped reads -2 for multiple reference sequences. All slices in this container must have a reference sequence id matching this value." itf8 "starting position on the reference" the alignment start position itf8 alignment span the length of the alignment itf8 number of records number of records in the container ltf8 record counter 1-based sequential index of records in the file/stream. ltf8 bases number of read bases itf8 number of blocks the total number of blocks in this container array<itf8> landmarks "the locations of slices in this container as byte offsets from the end of this container header, used for random access indexing. For sequence data containers, the landmark count must equal the slice count. Since the block before the first slice is the compression header, landmarks[0] is equal to the byte length of the compression header." int crc32 CRC32 hash of the all the preceding bytes in the container. byte[ blocks The blocks contained within the container.| Data type | Name | Value | | :---: | :---: | :---: | | int32 | length | the sum of the lengths of all blocks in this container <br> (headers and data) and any padding bytes (CRAM header <br> container only); equal to the total byte length of the <br> container minus the byte length of this header structure | | itf8 | reference sequence id | reference sequence identifier or <br> -1 for unmapped reads <br> -2 for multiple reference sequences. <br> All slices in this container must have a reference sequence <br> id matching this value. | | itf8 | starting position on the <br> reference | the alignment start position | | itf8 | alignment span | the length of the alignment | | itf8 | number of records | number of records in the container | | ltf8 | record counter | 1-based sequential index of records in the file/stream. | | ltf8 | bases | number of read bases | | itf8 | number of blocks | the total number of blocks in this container | | array<itf8> | landmarks | the locations of slices in this container as byte offsets from <br> the end of this container header, used for random access <br> indexing. For sequence data containers, the landmark <br> count must equal the slice count. <br> Since the block before the first slice is the compression <br> header, landmarks[0] is equal to the byte length of the <br> compression header. | | int | crc32 | CRC32 hash of the all the preceding bytes in the container. | | byte[ | blocks | The blocks contained within the container. |
In the initial CRAM header container, the reference sequence id, starting position on the reference, and alignment span fields must be ignored when reading. The landmarks array is optional for the CRAM header, but if it exists it should point to block offsets instead of slices, with the first block containing the textual header.
在初始 CRAM 标头容器中,读取时必须忽略参考序列 ID、参考起始位置和排列跨度字段。对于 CRAM 标头来说,地标数组是可选的,但如果存在,则应指向块偏移而不是片段,第一个块包含文本标头。
In data containers specifying unmapped reads or multiple reference sequences (i.e. reference sequence id < 0 < 0 < 0<0 ), the starting position on the reference and alignment span fields must be ignored when reading. When writing, it is recommended to set each of these ignored fields to the value 0 .
在指定未映射读数或多个参考序列(即参考序列 ID < 0 < 0 < 0<0 )的数据容器中,读取时必须忽略参考和排列跨度字段的起始位置。写入时,建议将每个忽略字段的值设置为 0。

7.1 CRAM header container
7.1 CRAM 标头容器

The first container in a CRAM file contains a textual header in one or more blocks. See section 8.3 for more details on the layout of data within these blocks and constraints applied to the contents of the SAM header.
CRAM 文件的第一个容器包含一个或多个块中的文本标头。有关这些块中的数据布局以及适用于 SAM 标头内容的限制的更多详情,请参阅第 8.3 节。
The landmarks field of the container header structure may be used to indicate the offsets of the blocks used in the header container. These may optionally be omitted by specifying an array size of zero.
容器标头结构的地标字段可用于指示标头容器中使用的块的偏移量。如果指定数组大小为零,则可以省略这些偏移。

8 Block structure 8 块结构

Containers consist of one or more blocks. Block compression is applied independently and in addition to any encodings used to compress data within the block. The block have the following header structure with the data stored in the ‘block data’ field:
容器由一个或多个数据块组成。除了用于压缩块内数据的编码外,块压缩是独立应用的。数据块具有以下标题结构,数据存储在 "数据块数据 "字段中:
Data type 数据类型 Name 名称 Value 价值
byte 字节 method 方法 the block compression method (and first CRAM version):
块压缩方法(以及第一个 CRAM 版本):
0: raw (none)* 0:未处理(无)*
1: gzip
2: bzip2 (v2.0)
3: lzma (v3.0)
4: rans4x8 (v3.0)
5: rans4x16 (v3.1)
6: adaptive arithmetic coder (v3.1)
6:自适应算术编码器(V3.1)
7: fqzcomp (v3.1)
8: name tokeniser (v3.1) 8:名称标记器(v3.1)
byte 字节 block content type id 块内容类型 id the block content type identifier
块内容类型标识符
itf8 size in bytes* 大小(字节)* the block content identifier used to associate external data
用于关联外部数据的块内容标识符
raw size in bytes* 原始大小(以字节为单位)* blocks with data series 数据系列块
itf8 block data 块数据 size of the block data after applying block compression
应用块压缩后的块数据大小
itf8 the data stored in the before applying block compression
在应用数据块压缩之前存储的数据
byte[] ・ bit stream of CRAM records (core data block)
・ CRAM 记录的比特流(核心数据块)
\bullet byte stream (external data block)
\bullet 字节流(外部数据块)
CRC32 additional fields ( header blocks)
附加字段(标头块)
byte[4] 字节[4] CRC32 hash value for all preceding bytes in the block
数据块中前面所有字节的 CRC32 哈希值
Data type Name Value byte method the block compression method (and first CRAM version): 0: raw (none)* 1: gzip 2: bzip2 (v2.0) 3: lzma (v3.0) 4: rans4x8 (v3.0) 5: rans4x16 (v3.1) 6: adaptive arithmetic coder (v3.1) 7: fqzcomp (v3.1) 8: name tokeniser (v3.1) byte block content type id the block content type identifier itf8 size in bytes* the block content identifier used to associate external data raw size in bytes* blocks with data series itf8 block data size of the block data after applying block compression itf8 the data stored in the before applying block compression byte[] ・ bit stream of CRAM records (core data block) ∙ byte stream (external data block) CRC32 additional fields ( header blocks) byte[4] CRC32 hash value for all preceding bytes in the block | Data type | Name | Value | | :--- | :--- | :--- | | byte | method | the block compression method (and first CRAM version): | | | | 0: raw (none)* | | | | 1: gzip | | | | 2: bzip2 (v2.0) | | | | 3: lzma (v3.0) | | | | 4: rans4x8 (v3.0) | | | | 5: rans4x16 (v3.1) | | | | 6: adaptive arithmetic coder (v3.1) | | | | 7: fqzcomp (v3.1) | | | | 8: name tokeniser (v3.1) | | byte | block content type id | the block content type identifier | | itf8 | size in bytes* | the block content identifier used to associate external data | | | raw size in bytes* | blocks with data series | | itf8 | block data | size of the block data after applying block compression | | itf8 | | the data stored in the before applying block compression | | byte[] | ・ bit stream of CRAM records (core data block) | | | | | $\bullet$ byte stream (external data block) | | | CRC32 | additional fields ( header blocks) | | byte[4] | CRC32 hash value for all preceding bytes in the block | |
  • Note on raw method: both compressed and raw sizes must be set to the same value.
    原始方法注意事项:压缩大小和原始大小必须设置为相同的值。
Empty blocks may occur in the files. Blocks with a raw (uncompressed) size of zero are treated as empty, irrespective of their “method” byte. This is equivalent to interpreting them as having method zero (raw) and compressed size of zero.
文件中可能会出现空块。原始(未压缩)大小为零的块将被视为空块,与其 "方法 "字节无关。这相当于将它们解释为方法为零(原始)、压缩后大小为零。

8.1 Block content types 8.1 块内容类型

CRAM has the following block content types:
CRAM 具有以下块内容类型:
Block content type 区块内容类型
 块内容类型 id
Block
content
type id
Block content type id| Block | | :--- | | content | | type id |
Name 名称 Contents 目录
FILE_HEADER 0 CRAM header block CRAM 标头块 CRAM header CRAM 标头
COMPRESSION_HEADER 1 Compression header block 压缩头块 See specific section 参见具体章节
SLICE_HEADER a ^("a "){ }^{\text {a }} 2 Slice header block 切片头块 See specific section 参见具体章节
3 reserved 矜持
EXTERNAL_DATA 4 external data block 外部数据块

外部编码产生的数据
data produced by
external encodings
data produced by external encodings| data produced by | | :--- | | external encodings |
CORE_DATA 5 core data block 核心数据块

除外部编码外的所有编码的比特流
bit stream of all
encodings except for
external encodings
bit stream of all encodings except for external encodings| bit stream of all | | :--- | | encodings except for | | external encodings |
Block content type "Block content type id" Name Contents FILE_HEADER 0 CRAM header block CRAM header COMPRESSION_HEADER 1 Compression header block See specific section SLICE_HEADER ^("a ") 2 Slice header block See specific section 3 reserved EXTERNAL_DATA 4 external data block "data produced by external encodings" CORE_DATA 5 core data block "bit stream of all encodings except for external encodings"| Block content type | Block <br> content <br> type id | Name | Contents | | :--- | :--- | :--- | :--- | | FILE_HEADER | 0 | CRAM header block | CRAM header | | COMPRESSION_HEADER | 1 | Compression header block | See specific section | | SLICE_HEADER ${ }^{\text {a }}$ | 2 | Slice header block | See specific section | | | 3 | | reserved | | EXTERNAL_DATA | 4 | external data block | data produced by <br> external encodings | | CORE_DATA | 5 | core data block | bit stream of all <br> encodings except for <br> external encodings |

8.2 Block content id 8.2 区块内容 ID

Block content id is used to distinguish between external blocks in the same slice. Each external encoding has an id parameter which must be one of the external block content ids. For external blocks the content id is a positive integer. For all other blocks content id should be 0 . Consequently, all external encodings must not use content id less than 1 .
块内容 id 用于区分同一片段中的外部块。每个外部编码都有一个 id 参数,它必须是外部块内容 id 之一。对于外部区块,内容 id 是一个正整数。对于所有其他区块,内容 id 应为 0。因此,所有外部编码不得使用小于 1 的内容 id。

Data blocks 数据块

Data is stored in data blocks. There are two types of data blocks: core data blocks and external data blocks. The difference between core and external data blocks is that core data blocks consist of data series that are compressed using bit encodings while the external data blocks are byte compressed. One core data block and any number of external data blocks are associated with each slice.
数据存储在数据块中。数据块有两种类型:核心数据块和外部数据块。核心数据块和外部数据块的区别在于,核心数据块由使用位编码压缩的数据序列组成,而外部数据块则是字节压缩的。每个片段关联一个核心数据块和任意数量的外部数据块。

Writing to and reading from core and external data blocks is organised through CRAM records. Each data series is associated with an encoding. In case of external encodings the block content id is used to identify the block where the data series is stored. Please note that external blocks can have multiple data series associated with them; in this case the values from these data series will be interleaved.
对核心和外部数据块的写入和读取是通过 CRAM 记录来组织的。每个数据序列都与编码相关联。在外部编码的情况下,数据块内容 ID 用于识别存储数据序列的数据块。请注意,外部数据块可以有多个与之相关的数据序列;在这种情况下,这些数据序列的值将交错排列。

8.3 CRAM header block(s) 8.3 CRAM 标头块

The SAM header is stored in the first block of the CRAM header container (see section 7.1). This block may be uncompressed or gzip compressed only. This block is followed by zero or more uncompressed expansion blocks. If present, these permit in-place editing of the CRAM header, allowing it to grow or shrink with a compensatory size change applied to the subsequence expansion block, avoiding the need to rewrite the remainder of the file. The contents of any expansion blocks should be zero bytes (nul characters).
SAM 标头存储在 CRAM 标头容器的第一个块中(见第 7.1 节)。该块可以是未压缩的,也可以是 gzip 压缩的。该块之后是零个或多个未压缩的扩展块。如果存在这些扩展块,就可以对 CRAM 标头进行就地编辑,通过对子扩展块进行补偿性大小更改来实现标头的增大或缩小,从而避免重写文件的其余部分。任何扩展块的内容都应为零字节(nul 字符)。

The format of the initial SAM header block is a 32-bit little-endian integer holding the length of the text of the SAM header, minus nul-termination bytes, followed by the text itself. Although 32-bit, the maximum permitted value is 2 31 2 31 2^(31)2^{31}, and all lengths must be positive.
SAM 首部数据块的格式是一个 32 位小二进制整数,其长度为 SAM 首部文本的长度减去无效终止字节,然后是文本本身。虽然是 32 位,但允许的最大值是 2 31 2 31 2^(31)2^{31} ,而且所有长度都必须是正数。

The following constraints apply to the SAM header text:
以下限制适用于 SAM 标头文本:
  • The SQ:MD5 checksum is required unless the reference sequence has been embedded into the file.
    除非参考序列已嵌入文件,否则需要 SQ:MD5 校验和。

8.4 Compression header block
8.4 压缩头块

The compression header block consists of 3 parts: preservation map, data series encoding map and tag encoding map.
压缩头块由 3 部分组成:保存图、数据序列编码图和标签编码图。

Preservation map 保护地图

The preservation map contains information about which data was preserved in the CRAM file. It is stored as a map with byte[2] keys:
保存映射包含 CRAM 文件中哪些数据被保存的信息。它以字节[2]键的映射形式存储:
Key 钥匙 Value data type 值数据类型 Name 名称 Value 价值
RN bool read names included 阅读名称包括 true if read names are preserved for all reads
如果所有读取都保留读取名称,则为 true
AP bool AP data series delta AP 数据系列 delta true if AP data series is delta, false otherwise
如果 AP 数据序列为 delta,则为 true,否则为 false
RR bool reference required 所需参考资料

如果需要参考序列才能完全恢复数据,则为 true
true if reference sequence is required to restore
the data completely
true if reference sequence is required to restore the data completely| true if reference sequence is required to restore | | :--- | | the data completely |
SM byte[5] 字节[5] substitution matrix 置换矩阵 substitution matrix 置换矩阵
TD array<byte> 数组<byte> tag ids dictionary 标签 ID 词典 a list of lists of tag ids, see tag encoding section
标签 ID 列表,参见标签编码部分
Key Value data type Name Value RN bool read names included true if read names are preserved for all reads AP bool AP data series delta true if AP data series is delta, false otherwise RR bool reference required "true if reference sequence is required to restore the data completely" SM byte[5] substitution matrix substitution matrix TD array<byte> tag ids dictionary a list of lists of tag ids, see tag encoding section| Key | Value data type | Name | Value | | :--- | :--- | :--- | :--- | | RN | bool | read names included | true if read names are preserved for all reads | | AP | bool | AP data series delta | true if AP data series is delta, false otherwise | | RR | bool | reference required | true if reference sequence is required to restore <br> the data completely | | SM | byte[5] | substitution matrix | substitution matrix | | TD | array<byte> | tag ids dictionary | a list of lists of tag ids, see tag encoding section |
The boolean values are optional, defaulting to true when absent, although it is recommended to explicitly set them. SM and TD are mandatory.
布尔值为可选项,不存在时默认为 true,但建议明确设置。SM 和 TD 为强制值。

Data series encodings 数据系列编码

Each data series has an encoding. These encoding are stored in a map with byte[2] keys and are decoded in approximately this order 2 2 ^(2){ }^{2} :
每个数据序列都有一个编码。这些编码存储在一个以字节[2]为键的映射中,解码顺序大致为 2 2 ^(2){ }^{2}
Key 钥匙 Value data type 值数据类型 Name 名称 Value 价值
BF encoding<int> 编码<int> BAM bit flags BAM 位标志 see separate section 见单独章节
CF encoding<int> 编码<int> CRAM bit flags CRAM 位标志 see specific section 参见具体章节
RI encoding<int> 编码<int> reference id 参考编号 record reference id from the SAM file header
SAM 文件头中的记录引用标识
RL encoding<int> 编码<int> read lengths 阅读长度 read lengths 阅读长度
AP encoding<int> 编码<int> in-seq positions 序列内位置

如果 AP-Delta = true:与前一条记录中的 AP 值的对齐起始三角洲值为 0。注意该三角洲值可能为负,例如在多参考分片中切换参考时。如果 AP-Delta = false:直接编码对齐起始位置。
if AP-Delta = true: 0-based alignment start
delta from the AP value in the previous record.
Note this delta may be negative, for example
when switching references in a multi-reference
slice. When the record is the first in the slice, the
previous position used is the slice alignment-start
field (hence the first delta should be zero for
single-reference slices, or the AP value itself for
multi-reference slices).
if AP-Delta = false: encodes the alignment start
position directly
if AP-Delta = true: 0-based alignment start delta from the AP value in the previous record. Note this delta may be negative, for example when switching references in a multi-reference slice. When the record is the first in the slice, the previous position used is the slice alignment-start field (hence the first delta should be zero for single-reference slices, or the AP value itself for multi-reference slices). if AP-Delta = false: encodes the alignment start position directly| if AP-Delta = true: 0-based alignment start | | :--- | | delta from the AP value in the previous record. | | Note this delta may be negative, for example | | when switching references in a multi-reference | | slice. When the record is the first in the slice, the | | previous position used is the slice alignment-start | | field (hence the first delta should be zero for | | single-reference slices, or the AP value itself for | | multi-reference slices). | | if AP-Delta = false: encodes the alignment start | | position directly |
RG encoding<int> 编码<int> read groups 阅读小组

读取组。特殊值"-1 "代表无组。
read groups. Special value ' -1 ' stands for no
group.
read groups. Special value ' -1 ' stands for no group.| read groups. Special value ' -1 ' stands for no | | :--- | | group. |
RN a RN a RN^(a)\mathrm{RN}^{\mathrm{a}} encoding<byte[ ]> 编码<byte[ ]> read names 阅读名称 read names 阅读名称
MF encoding<int> 编码<int> next mate bit flags 下一个队友位标志 see specific section 参见具体章节
NS encoding<int> 编码<int>

下一个片段参考序列 ID
next fragment
reference sequence id
next fragment reference sequence id| next fragment | | :--- | | reference sequence id |
reference sequence ids for the next fragment
下一个片段的参考序列 ID
NP encoding<int> 编码<int>
 下一个对齐开始
next mate alignment
start
next mate alignment start| next mate alignment | | :--- | | start |
alignment positions for the next fragment
下一个片段的对齐位置
TS encoding<int> 编码<int> template size 模板尺寸 template sizes 模板尺寸
NF encoding<int> 编码<int>
 到下一个分段的距离
distance to next
fragment
distance to next fragment| distance to next | | :--- | | fragment |
number of records to skip to the next fragment b b ^(b){ }^{b}
跳转到下一个片段的记录数 b b ^(b){ }^{b}
TL C TL C TL^(C)\mathrm{TL}^{\mathrm{C}} encoding<int> 编码<int> tag ids 标签 id list of tag ids, see tag encoding section
标签 id 列表,参见标签编码部分
FN encoding<int> 编码<int>
 阅读次数
number of read
features
number of read features| number of read | | :--- | | features |
number of read features in each record
每条记录中的读取特征数
FC encoding<byte> 编码<byte> read features codes 读取功能代码 see separate section 见单独章节
FP encoding<int> 编码<int> in-read positions 读入位置

读取特征的位置;最后一个位置的正 delta 值(从零开始)
positions of the read features; a positive delta to
the last position (starting with zero)
positions of the read features; a positive delta to the last position (starting with zero)| positions of the read features; a positive delta to | | :--- | | the last position (starting with zero) |
DL encoding<int> 编码<int> deletion lengths 删除长度 base-pair deletion lengths
碱基对缺失长度
BB encoding<byte[]> 编码<byte[]> stretches of bases 基地 bases 基地
QQ encoding<byte[ ]> 编码<byte[ ]>

质量分数线的长度
stretches of quality
scores
stretches of quality scores| stretches of quality | | :--- | | scores |
quality scores 质量得分
BS encoding<byte> 编码<byte>
 基本替换代码
base substitution
codes
base substitution codes| base substitution | | :--- | | codes |
base substitution codes 碱基替换码
IN encoding<byte[]> 编码<byte[]> insertion 插入 inserted bases 插入式基座
RS encoding<int> 编码<int> reference skip length 参考跳读长度 number of skipped bases for the ' N ' read feature
N "读数特征的跳过碱基数
PD encoding<int> 编码<int> padding 衬垫 number of padded bases 垫底数量
HC encoding<int> 编码<int> hard clip 硬夹 number of hard clipped bases
硬剪切基数
SC encoding<byte[ ]> 编码<byte[ ]> soft clip 软夹 soft clipped bases 软剪裁底座
MQ encoding<int> 编码<int> mapping qualities 绘图质量 mapping quality scores 绘制质量分数
BA encoding<byte> 编码<byte> bases 基地 bases 基地
QS encoding<byte> 编码<byte> quality scores 质量得分 quality scores 质量得分
TC d TC d TC^(d)\mathrm{TC}^{\mathrm{d}} N/A 不适用 legacy field 遗留字段 to be ignored 置之不理
TN d TN d TN^(d)\mathrm{TN}^{\mathrm{d}} N/A 不适用 legacy field 遗留字段 to be ignored 置之不理
Key Value data type Name Value BF encoding<int> BAM bit flags see separate section CF encoding<int> CRAM bit flags see specific section RI encoding<int> reference id record reference id from the SAM file header RL encoding<int> read lengths read lengths AP encoding<int> in-seq positions "if AP-Delta = true: 0-based alignment start delta from the AP value in the previous record. Note this delta may be negative, for example when switching references in a multi-reference slice. When the record is the first in the slice, the previous position used is the slice alignment-start field (hence the first delta should be zero for single-reference slices, or the AP value itself for multi-reference slices). if AP-Delta = false: encodes the alignment start position directly" RG encoding<int> read groups "read groups. Special value ' -1 ' stands for no group." RN^(a) encoding<byte[ ]> read names read names MF encoding<int> next mate bit flags see specific section NS encoding<int> "next fragment reference sequence id" reference sequence ids for the next fragment NP encoding<int> "next mate alignment start" alignment positions for the next fragment TS encoding<int> template size template sizes NF encoding<int> "distance to next fragment" number of records to skip to the next fragment ^(b) TL^(C) encoding<int> tag ids list of tag ids, see tag encoding section FN encoding<int> "number of read features" number of read features in each record FC encoding<byte> read features codes see separate section FP encoding<int> in-read positions "positions of the read features; a positive delta to the last position (starting with zero)" DL encoding<int> deletion lengths base-pair deletion lengths BB encoding<byte[]> stretches of bases bases QQ encoding<byte[ ]> "stretches of quality scores" quality scores BS encoding<byte> "base substitution codes" base substitution codes IN encoding<byte[]> insertion inserted bases RS encoding<int> reference skip length number of skipped bases for the ' N ' read feature PD encoding<int> padding number of padded bases HC encoding<int> hard clip number of hard clipped bases SC encoding<byte[ ]> soft clip soft clipped bases MQ encoding<int> mapping qualities mapping quality scores BA encoding<byte> bases bases QS encoding<byte> quality scores quality scores TC^(d) N/A legacy field to be ignored TN^(d) N/A legacy field to be ignored| Key | Value data type | Name | Value | | :---: | :---: | :---: | :---: | | BF | encoding<int> | BAM bit flags | see separate section | | CF | encoding<int> | CRAM bit flags | see specific section | | RI | encoding<int> | reference id | record reference id from the SAM file header | | RL | encoding<int> | read lengths | read lengths | | AP | encoding<int> | in-seq positions | if AP-Delta = true: 0-based alignment start <br> delta from the AP value in the previous record. <br> Note this delta may be negative, for example <br> when switching references in a multi-reference <br> slice. When the record is the first in the slice, the <br> previous position used is the slice alignment-start <br> field (hence the first delta should be zero for <br> single-reference slices, or the AP value itself for <br> multi-reference slices). <br> if AP-Delta = false: encodes the alignment start <br> position directly | | RG | encoding<int> | read groups | read groups. Special value ' -1 ' stands for no <br> group. | | $\mathrm{RN}^{\mathrm{a}}$ | encoding<byte[ ]> | read names | read names | | MF | encoding<int> | next mate bit flags | see specific section | | NS | encoding<int> | next fragment <br> reference sequence id | reference sequence ids for the next fragment | | NP | encoding<int> | next mate alignment <br> start | alignment positions for the next fragment | | TS | encoding<int> | template size | template sizes | | NF | encoding<int> | distance to next <br> fragment | number of records to skip to the next fragment ${ }^{b}$ | | $\mathrm{TL}^{\mathrm{C}}$ | encoding<int> | tag ids | list of tag ids, see tag encoding section | | FN | encoding<int> | number of read <br> features | number of read features in each record | | FC | encoding<byte> | read features codes | see separate section | | FP | encoding<int> | in-read positions | positions of the read features; a positive delta to <br> the last position (starting with zero) | | DL | encoding<int> | deletion lengths | base-pair deletion lengths | | BB | encoding<byte[]> | stretches of bases | bases | | QQ | encoding<byte[ ]> | stretches of quality <br> scores | quality scores | | BS | encoding<byte> | base substitution <br> codes | base substitution codes | | IN | encoding<byte[]> | insertion | inserted bases | | RS | encoding<int> | reference skip length | number of skipped bases for the ' N ' read feature | | PD | encoding<int> | padding | number of padded bases | | HC | encoding<int> | hard clip | number of hard clipped bases | | SC | encoding<byte[ ]> | soft clip | soft clipped bases | | MQ | encoding<int> | mapping qualities | mapping quality scores | | BA | encoding<byte> | bases | bases | | QS | encoding<byte> | quality scores | quality scores | | $\mathrm{TC}^{\mathrm{d}}$ | N/A | legacy field | to be ignored | | $\mathrm{TN}^{\mathrm{d}}$ | N/A | legacy field | to be ignored |
a a ^(a){ }^{a} Note RN this is decoded after MF if the record is detached from the mate and we are attempting to auto-generate read names.
a a ^(a){ }^{a} 请注意,如果记录与队友分离,且我们正试图自动生成读取名称,则 RN 会在 MF 之后解码。

b b ^(b){ }^{\mathrm{b}} The count is reset for each slice so NF can only refer to a record later within this slice.
b b ^(b){ }^{\mathrm{b}} 每个片段的计数都会被重置,因此 NF 只能在该片段的后面引用记录。

c c ^(c){ }^{c} TL is followed by decoding the tag values themselves, in order of appearance in the tag dictionary.
c c ^(c){ }^{c} TL 之后,将按照标签字典中出现的顺序对标签值本身进行解码。

d TC d TC ^(d)TC{ }^{\mathrm{d}} \mathrm{TC} and TN are legacy data series from CRAM 1.0. They have no function in CRAM 3.0 and should not be present. However some implementations do output them and decoders must silently skip these fields. It is illegal for TC and TN to contain any data values, although there may be empty blocks associated with them.
d TC d TC ^(d)TC{ }^{\mathrm{d}} \mathrm{TC} 和 TN 是 CRAM 1.0 中的遗留数据系列。它们在 CRAM 3.0 中没有任何功能,不应出现。不过,有些实现确实会输出它们,解码器必须静默地跳过这些字段。TC 和 TN 不允许包含任何数据值,尽管可能存在与之相关的空块。

Tag encodings 标签编码

The tag dictionary (TD) describes the unique combinations of tag id / type that occur on each alignment record. For example if we search the id / types present in each record and find only two combinations - X1:i BC:Z SA:Z: and X1:i: BC:Z - then we have two dictionary entries in the TD map.
标签字典 (TD) 描述了每个对齐记录中出现的标签 id / 类型的唯一组合。例如,如果我们搜索每条记录中出现的标识/类型,发现只有两种组合--X1:i BC:Z SA:Z: 和 X1:i: BC:Z --那么在 TD 地图中就有两个字典条目。

Let L i = { T i 0 , T i 1 , , T i x } L i = T i 0 , T i 1 , , T i x L_(i)={T_(i0),T_(i1),dots,T_(ix)}L_{i}=\left\{T_{i 0}, T_{i 1}, \ldots, T_{i x}\right\} be a list of all tag ids for a record R i R i R_(i)R_{i}, where i i ii is the sequential record index and T i j T i j T_(ij)T_{i j} denotes j j jj-th tag id in the record. The list of unique L i L i L_(i)L_{i} is stored as the TD value in the preservation map. Maintaining the order is not a requirement for encoders (hence “combinations”), but it is permissible and thus different permutations, each encoded with their own elements in TD, should be supported by the decoder. Each L i L i L_(i)L_{i} element in TD is assigned a sequential integer number starting with 0 . These integer numbers are referred to by the TL data series. Using TD, an integer from the TL data series can be mapped back into a list of tag ids. Thus per alignment record we only need to store tag values and not their ids and types.
假设 L i = { T i 0 , T i 1 , , T i x } L i = T i 0 , T i 1 , , T i x L_(i)={T_(i0),T_(i1),dots,T_(ix)}L_{i}=\left\{T_{i 0}, T_{i 1}, \ldots, T_{i x}\right\} 是记录 R i R i R_(i)R_{i} 的所有标记 ID 列表,其中 i i ii 是顺序记录索引, T i j T i j T_(ij)T_{i j} 表示记录中的 j j jj 个标记 ID。唯一的 L i L i L_(i)L_{i} 列表作为 TD 值保存在保存映射中。保持顺序不是编码器的要求(因此称为 "组合"),但它是允许的,因此解码器应支持不同的排列,每种排列在 TD 中都有自己的编码元素。TD 中的每个 L i L i L_(i)L_{i} 元素都被分配了一个从 0 开始的连续整数。这些整数由 TL 数据序列表示。使用 TD,可以将 TL 数据序列中的整数映射回标签 ID 列表。因此,我们只需存储每个对齐记录的标签值,而无需存储标签 id 和类型。

The TD is written as a byte array consisting of L i L i L_(i)L_{i} values separated with 0 0 \\0\backslash 0. Each L i L i L_(i)L_{i} value is written as a concatenation of 3 byte T i j T i j T_(ij)T_{i j} elements: tag id followed by BAM tag type code (one of A, c, C, s, S, i, I, f, Z, H or B , as described in the SAM specification). For example the TD for tag lists X1:i BC:Z SA:Z and X1:i BC:Z may be encoded as X1CBCZSAZ 0 X 1 CBCZ 0 0 X 1 CBCZ 0 \\0X1CBCZ\\0\backslash 0 \mathrm{X} 1 \mathrm{CBCZ} \backslash 0, with X 1 C indicating a 1 byte unsigned value for tag X 1 .
TD 被写成一个字节数组,由 L i L i L_(i)L_{i} 值组成,并用 0 0 \\0\backslash 0 分隔。每个 L i L i L_(i)L_{i} 值都以 3 个字节 T i j T i j T_(ij)T_{i j} 元素的连接形式写入:标签 ID,后跟 BAM 标签类型代码(A、c、C、s、S、i、I、f、Z、H 或 B 之一,如 SAM 规范所述)。例如,标签列表 X1:i BC:Z SA:Z 和 X1:i BC:Z 的 TD 可以编码为 X1CBCZSAZ 0 X 1 CBCZ 0 0 X 1 CBCZ 0 \\0X1CBCZ\\0\backslash 0 \mathrm{X} 1 \mathrm{CBCZ} \backslash 0 ,其中 X 1 C 表示标签 X 1 的 1 字节无符号值。

Tag values 标签值

The encodings used for different tags are stored in a map. The key is 3 bytes formed from the BAM tag id and type code, matching the TD dictionary described above. Unlike the Data Series Encoding Map, the key is stored in the map as an ITF8 encoded integer, constructed using (char 1 << 16 ) + ( 1 << 16 ) + ( 1<<16)+(1<<16)+( char 2 << 8 ) + 2 << 8 ) + 2<<8)+2<<8)+ type. For example, the 3 -byte representation of OQ:Z is { 0 x 4 F , 0 x 51 , 0 × 5 A } { 0 x 4 F , 0 x 51 , 0 × 5 A } {0x4F,0x51,0xx5A}\{0 \mathrm{x} 4 \mathrm{~F}, 0 \mathrm{x} 51,0 \times 5 \mathrm{~A}\} and these bytes are interpreted as the integer key 0 x 004 F 515 A , leading to an ITF8 byte stream { 0 xE 0 , 0 x 4 F , 0 x 51 , 0 x 5 A } { 0 xE 0 , 0 x 4 F , 0 x 51 , 0 x 5 A } {0xE0,0x4F,0x51,0x5A}\{0 \mathrm{xE} 0,0 \mathrm{x} 4 \mathrm{~F}, 0 \mathrm{x} 51,0 \mathrm{x} 5 \mathrm{~A}\}.
不同标记使用的编码存储在一个映射中。键是由 BAM 标记 ID 和类型代码组成的 3 个字节,与上述 TD 字典相匹配。与数据系列编码映射表不同的是,键以 ITF8 编码整数形式存储在映射表中,使用(char 1 << 16 ) + ( 1 << 16 ) + ( 1<<16)+(1<<16)+( char 2 << 8 ) + 2 << 8 ) + 2<<8)+2<<8)+ 类型)构建。例如,OQ:Z 的 3 字节表示为 { 0 x 4 F , 0 x 51 , 0 × 5 A } { 0 x 4 F , 0 x 51 , 0 × 5 A } {0x4F,0x51,0xx5A}\{0 \mathrm{x} 4 \mathrm{~F}, 0 \mathrm{x} 51,0 \times 5 \mathrm{~A}\} ,这些字节被解释为整数键 0 x 004 F 515 A,从而产生一个 ITF8 字节流 { 0 xE 0 , 0 x 4 F , 0 x 51 , 0 x 5 A } { 0 xE 0 , 0 x 4 F , 0 x 51 , 0 x 5 A } {0xE0,0x4F,0x51,0x5A}\{0 \mathrm{xE} 0,0 \mathrm{x} 4 \mathrm{~F}, 0 \mathrm{x} 51,0 \mathrm{x} 5 \mathrm{~A}\}