public abstract class WarcValidatorBase extends WarcConstants
| Modifier and Type | Field and Description |
|---|---|
protected int[][] |
field_policy
A (Warc-Types x Warc-Header-Fields) matrix used for policy validation.
|
int[] |
usedOnVersion
Version denoted as an integer array.
|
CONTENT_TYPE_FORMAT, CONTENT_TYPE_METADATA, CT_APP_WARC_FIELDS, endMark, FDT_CONTENTTYPE, FDT_DATE, FDT_DIGEST, FDT_IDX_STRINGS, FDT_INETADDRESS, FDT_INTEGER, FDT_LONG, FDT_STRING, FDT_URI, fieldNameIdxMap, fieldNamesRepeatableLookup, FN_CONTENT_LENGTH, FN_CONTENT_TYPE, FN_IDX_CONTENT_LENGTH, FN_IDX_CONTENT_TYPE, FN_IDX_DT, FN_IDX_STRINGS, FN_IDX_WARC_BLOCK_DIGEST, FN_IDX_WARC_CONCURRENT_TO, FN_IDX_WARC_DATE, FN_IDX_WARC_FILENAME, FN_IDX_WARC_IDENTIFIED_PAYLOAD_TYPE, FN_IDX_WARC_IP_ADDRESS, FN_IDX_WARC_PAYLOAD_DIGEST, FN_IDX_WARC_PROFILE, FN_IDX_WARC_RECORD_ID, FN_IDX_WARC_REFERS_TO, FN_IDX_WARC_REFERS_TO_DATE, FN_IDX_WARC_REFERS_TO_TARGET_URI, FN_IDX_WARC_SEGMENT_NUMBER, FN_IDX_WARC_SEGMENT_ORIGIN_ID, FN_IDX_WARC_SEGMENT_TOTAL_LENGTH, FN_IDX_WARC_TARGET_URI, FN_IDX_WARC_TRUNCATED, FN_IDX_WARC_TYPE, FN_IDX_WARC_WARCINFO_ID, FN_INDEX_OF_LAST, FN_NUMBER, FN_WARC_BLOCK_DIGEST, FN_WARC_CONCURRENT_TO, FN_WARC_DATE, FN_WARC_FILENAME, FN_WARC_IDENTIFIED_PAYLOAD_TYPE, FN_WARC_IP_ADDRESS, FN_WARC_PAYLOAD_DIGEST, FN_WARC_PROFILE, FN_WARC_RECORD_ID, FN_WARC_REFERS_TO, FN_WARC_REFERS_TO_DATE, FN_WARC_REFERS_TO_TARGET_URI, FN_WARC_SEGMENT_NUMBER, FN_WARC_SEGMENT_ORIGIN_ID, FN_WARC_SEGMENT_TOTAL_LENGTH, FN_WARC_TARGET_URI, FN_WARC_TRUNCATED, FN_WARC_TYPE, FN_WARC_WARCINFO_ID, MEDIA_TYPE_METADATA, P_IDX_STRINGS, POLICY_IGNORE, POLICY_MANDATORY, POLICY_MAY, POLICY_MAY_NOT, POLICY_SHALL, POLICY_SHALL_NOT, profileIdxMap, recordTypeIdxMap, RT_CONTINUATION, RT_CONVERSION, RT_IDX_CONTINUATION, RT_IDX_CONVERSION, RT_IDX_METADATA, RT_IDX_REQUEST, RT_IDX_RESOURCE, RT_IDX_RESPONSE, RT_IDX_REVISIT, RT_IDX_STRINGS, RT_IDX_UNKNOWN, RT_IDX_WARCINFO, RT_INDEX_OF_LAST, RT_METADATA, RT_NUMBER, RT_REQUEST, RT_RESOURCE, RT_RESPONSE, RT_REVISIT, RT_WARCINFO, truncatedTypeIdxMap, TT_DISCONNECT, TT_IDX_DISCONNECT, TT_IDX_FUTURE_REASON, TT_IDX_LENGTH, TT_IDX_STRINGS, TT_IDX_TIME, TT_IDX_UNSPECIFIED, TT_LENGTH, TT_TIME, TT_UNSPECIFIED, WARC_DATE_FORMAT, WARC_DIGEST_FORMAT, WARC_MAGIC_HEADER, WARC_MIME_TYPE, WARC_PROFILE_IDX_UNKNOWN, WARC_RECORD_TRAILING_NEWLINES, WARC10_PROFILE_IDENTICAL_PAYLOAD_DIGEST, WARC10_PROFILE_IDX_IDENTICAL_PAYLOAD_DIGEST, WARC10_PROFILE_IDX_SERVER_NOT_MODIFIED, WARC10_PROFILE_SERVER_NOT_MODIFIED, WARC11_PROFILE_IDENTICAL_PAYLOAD_DIGEST, WARC11_PROFILE_IDX_IDENTICAL_PAYLOAD_DIGEST, WARC11_PROFILE_IDX_SERVER_NOT_MODIFIED, WARC11_PROFILE_SERVER_NOT_MODIFIED| Modifier | Constructor and Description |
|---|---|
protected |
WarcValidatorBase()
Internal constructor.
|
| Modifier and Type | Method and Description |
|---|---|
void |
checkFieldPolicy(int recordType,
int fieldType,
java.lang.Object fieldObj,
java.lang.String valueStr,
Diagnostics diagnostics)
Given a WARC record type and a WARC field looks up the policy in a
matrix build from the WARC ISO standard.
|
protected void |
checkFields(WarcHeader header)
Validate the WARC header relative to the WARC-Type and according to the
WARC ISO standard.
|
void |
setVersions(int[] arr)
Set the version this validator is used on.
|
abstract void |
setWarcHeaderPolicyMatrix()
The following section initializes the policy matrix used to check the
usage of each known warc header line against each known warc record
type.
|
public int[] usedOnVersion
protected int[][] field_policy
protected WarcValidatorBase()
public void setVersions(int[] arr)
arr - integer array denoting a versionpublic abstract void setWarcHeaderPolicyMatrix()
protected void checkFields(WarcHeader header)
header - warc header to validatepublic void checkFieldPolicy(int recordType, int fieldType, java.lang.Object fieldObj, java.lang.String valueStr, Diagnostics diagnostics)
recordType - WARC record type idfieldType - WARC field type idfieldObj - WARC fieldvalueStr - WARC raw field valuediagnostics - diagnostics object used to record warnings/errorsCopyright © 2011–2023. All rights reserved.