Class URLParser

java.lang.Object
org.archive.url.URLParser
public class URLParser extends Object

  • Field Details

    • RFC2396REGEX

      public static final Pattern RFC2396REGEX
      RFC 2396-inspired regex. From the RFC Appendix B: 
       URI Generic Syntax                August 1998

 B. Parsing a URI Reference with a Regular Expression

 As described in Section 4.3, the generic URI syntax is not sufficient
 to disambiguate the components of some forms of URI.  Since the
 "greedy algorithm" described in that section is identical to the
 disambiguation method used by POSIX regular expressions, it is
 natural and commonplace to use a regular expression for parsing the
 potential four components and fragment identifier of a URI reference.

 The following line is the regular expression for breaking-down a URI
 reference into its components.

 ^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))?
  12            3  4          5       6  7        8 9

 The numbers in the second line above are only to assist readability;
 they indicate the reference points for each subexpression (i.e., each
 paired parenthesis).  We refer to the value matched for subexpression
 <n> as $<n>.  For example, matching the above expression to

 http://www.ics.uci.edu/pub/ietf/uri/#Related

 results in the following subexpression matches:

 $1 = http:
 $2 = http
 $3 = //www.ics.uci.edu
 $4 = www.ics.uci.edu
 $5 = /pub/ietf/uri/
 $6 = <undefined>
 $7 = <undefined>
 $8 = #Related
 $9 = Related

 where <undefined> indicates that the component is not present, as is
 the case for the query component in the above example.  Therefore, we
 can determine the value of the four components and fragment as

 scheme    = $2
 authority = $4
 path      = $5
 query     = $7
 fragment  = $9
      --

      Below differs from the rfc regex in that... (1) it has java escaping of regex characters (2) we allow a URI made of a fragment only (Added extra group so indexing is off by one after scheme). (3) scheme is limited to legal scheme characters

    • COMMERCIAL_AT

      public static final String COMMERCIAL_AT
      See Also:

    • PERCENT_SIGN

      public static final char PERCENT_SIGN
      See Also:

    • COLON

      public static final char COLON
      See Also:

    • STRAY_SPACING

      public static final String STRAY_SPACING
      See Also:

    • DNS_SCHEME

      public static final String DNS_SCHEME
      ARC/WARC specific DNS resolution record.
      See Also:

    • FILEDESC_SCHEME

      public static final String FILEDESC_SCHEME
      ARC header record.
      See Also:

    • WARCINFO_SCHEME

      public static final String WARCINFO_SCHEME
      WARC header record.
      See Also:

    • HTTP_SCHEME

      public static final String HTTP_SCHEME
      HTTP
      See Also:

    • HTTPS_SCHEME

      public static final String HTTPS_SCHEME
      HTTPS
      See Also:

    • FTP_SCHEME

      public static final String FTP_SCHEME
      FTP
      See Also:

    • MMS_SCHEME

      public static final String MMS_SCHEME
      MMS
      See Also:

    • RTSP_SCHEME

      public static final String RTSP_SCHEME
      RTSP
      See Also:

    • DEFAULT_SCHEME

      public static final String DEFAULT_SCHEME
      Default scheme to assume if unspecified. No context implied...
      See Also:

    • WAIS_SCHEME

      public static final String WAIS_SCHEME
      go brewster
      See Also:

    • ALL_SCHEMES

      public static final String[] ALL_SCHEMES
      array of static Strings for all "known" schemes

    • ALL_SCHEMES_PATTERN

      public static final Pattern ALL_SCHEMES_PATTERN

  • Constructor Details

    • URLParser

      public URLParser()

  • Method Details

    • urlToScheme

      public static String urlToScheme(String url)
      Attempt to find the scheme (http://, https://, etc) from a given URL.
      Parameters:
      url - URL String to parse for a scheme.
      Returns:
      the scheme, including trailing "://" if known, null otherwise.

    • addDefaultSchemeIfNeeded

      public static String addDefaultSchemeIfNeeded(String urlString)

    • parse

      public static HandyURL parse(String urlString) throws URISyntaxException
      Throws:
      URISyntaxException