pax_global_header00006660000000000000000000000064137551767100014526gustar00rootroot0000000000000052 comment=bf325e3161b30638c4e4ac11960c4e7658384e07 cabby-0.1.23/000077500000000000000000000000001375517671000126715ustar00rootroot00000000000000cabby-0.1.23/.coveragerc000066400000000000000000000000401375517671000150040ustar00rootroot00000000000000[run] omit = cabby/cli/*.py cabby-0.1.23/.dockerignore000066400000000000000000000013141375517671000153440ustar00rootroot00000000000000# Git .git/ # Byte-compiled / optimized / DLL files __pycache__/ *.py[cod] # C extensions *.so # Distribution / packaging .Python env/ build/ develop-eggs/ dist/ downloads/ eggs/ lib/ lib64/ parts/ sdist/ var/ *.egg-info/ .installed.cfg *.egg # PyInstaller # Usually these files are written by a python script from a template # before PyInstaller builds the exe, so as to inject date/other infos into it. *.manifest *.spec # Installer logs pip-log.txt pip-delete-this-directory.txt # Unit test / coverage reports htmlcov/ .tox/ .coverage .cache nosetests.xml coverage.xml # Translations *.mo *.pot # Django stuff: *.log # Sphinx documentation docs/_build/ # PyBuilder target/ # Editor files *.iml .idea cabby-0.1.23/.gitignore000066400000000000000000000012711375517671000146620ustar00rootroot00000000000000# Byte-compiled / optimized / DLL files __pycache__/ *.py[cod] # C extensions *.so # Distribution / packaging .Python env/ build/ develop-eggs/ dist/ downloads/ eggs/ lib/ lib64/ parts/ sdist/ var/ *.egg-info/ .installed.cfg *.egg # PyInstaller # Usually these files are written by a python script from a template # before PyInstaller builds the exe, so as to inject date/other infos into it. *.manifest *.spec # Installer logs pip-log.txt pip-delete-this-directory.txt # Unit test / coverage reports htmlcov/ .tox/ .coverage .cache nosetests.xml coverage.xml # Translations *.mo *.pot # Django stuff: *.log # Sphinx documentation docs/_build/ # PyBuilder target/ # Editor *.iml .idea cabby-0.1.23/.travis.yml000066400000000000000000000006271375517671000150070ustar00rootroot00000000000000branches: - master language: python dist: xenial python: - "2.7" - "3.5" - "3.6" - "3.7" install: - pip install coveralls -r requirements-dev.txt script: - py.test --cov cabby after_success: - coveralls matrix: include: - name: flake8 install: - pip install flake8 script: - flake8 after_success: - true # don't run coveralls for flake8 cabby-0.1.23/CHANGES.rst000066400000000000000000000072501375517671000144770ustar00rootroot00000000000000Changelog ========= 0.1.23 (2020-11-18) ------------------- * Only transparently refresh JWT token when using JWT auth (`pr#78 `_) 0.1.22 (2020-11-09) ------------------- * Fix bug where JWT would not be used if client.jwt_token is set directly (`pr#76 `_, `pr#77 `_) 0.1.21 (2020-11-09) ------------------- * Refresh JWT and retry request once on UNAUTHORIZED status message * Add discovery_url argument to create_client function 0.1.20 (2018-03-29) ------------------- * Only include relevant files in release packages. 0.1.19 (2018-03-29) ------------------- * Enable client key passphrase for JWT token authentication request (`pr#50 `_) 0.1.18 (2017-06-19) ------------------- * Dependencies upgraded (`changes `_). * ``timeout`` property that sets HTTP timeout added to the client class and CLI tools (thanks `@joarleymoraes `_). 0.1.17 (2017-04-03) ------------------- * Support for XML "huge trees" added. It can be disabled with environment variable. See :ref:`configuration_via_env_vars`. 0.1.16 (2016-10-31) ------------------- * Support for gzipped responses added. 0.1.15 (2016-10-19) ------------------- * Issue with unrecognized TLS key password is fixed. 0.1.14 (2016-10-19) ------------------- * Issue when Cabby was assuming port 80 for HTTPS URLs is fixed. 0.1.13 (2016-10-18) ------------------- * Cabby will always return content block body as bytes. * JWT token caching added. * Added support for local TLS CA files. * ``--port`` option in CLI commands gets a higher priority than port extracted from provided ``--path``. * Docker file moved to ``alpine`` as a base, shaving off 520MB from the total size of the container. 0.1.12 (2016-06-13) ------------------- * fix for incorrect connection error handling issue, that was affecting invalid connections with tls and key password. 0.1.11 (2016-04-27) ------------------- * Documentation fixes. 0.1.10 (2015-12-29) ------------------- * Removing incorrect assumption that auth details are always present. 0.1.9 (2015-12-04) ------------------ * Own implementation of TAXII transport logic added, using `Requests `_ library for most of the requests. * Added ``taxii-proxy`` CLI command to allow funneling of data from one taxii server to another. * Various bug fixes. 0.1.7 (2015-09-18) ------------------ * Fix for XML stream parser issue related to a race condition in libxml. 0.1.6 (2015-08-11) ------------------ * Missing dependencies added to setup.py. 0.1.5 (2015-08-10) ------------------ * Added Python 3 compatibility. * XML stream parsing for Poll Response messages. * Bugfixes. 0.1.4 (2015-07-24) ------------------ * ``--bindings`` option added for ``taxii-poll`` CLI command. * Pagination issue, triggered during processing of paged Poll response, was fixed. * CLI datetime parameters have UTC timezone by default. * JWT based authentication method added. * Multiple naming, style and bug fixes. 0.1.3 (2015-04-08) ------------------ * Workaround for libtaxii issue #186 (wrapping incorrect response in Status Message) has been added. * Tests improved. 0.1.2 (2015-03-31) ------------------ * Issue with proxy arguments being ignored is fixed. * Issue with poll results print in CLI referencing wrong entity is fixed. * Wording and style fixes. 0.1.1 (2015-03-26) ------------------ * Tidying up packaging and distribution related configuration. 0.1.0 (2015-03-26) ------------------ * Initial release. cabby-0.1.23/Dockerfile000066400000000000000000000005771375517671000146740ustar00rootroot00000000000000FROM python:3-slim-stretch LABEL maintainer="EclecticIQ " RUN python3 -m venv --system-site-packages /venv ENV PATH=/venv/bin:$PATH COPY ./requirements.txt ./requirements-dev.txt /cabby/ RUN pip install -r /cabby/requirements-dev.txt COPY . /cabby RUN pip install -e /cabby RUN sh -c "cat /cabby/docker-help.sh >> /root/.bashrc" CMD ["/cabby/docker-help.sh"] cabby-0.1.23/LICENSE.rst000066400000000000000000000030211375517671000145010ustar00rootroot00000000000000License ======= Copyright © 2016, EclecticIQ All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of the author nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. *(This is the OSI approved 3-clause "New BSD License".)* cabby-0.1.23/MANIFEST.in000066400000000000000000000002061375517671000144250ustar00rootroot00000000000000include LICENSE.rst include CHANGES.rst include README.rst include requirements.txt recursive-include docs *.rst global-exclude *.pyc cabby-0.1.23/README.rst000066400000000000000000000026521375517671000143650ustar00rootroot00000000000000cabby ===== Python TAXII client implementation from `EclecticIQ `_. :Source: https://github.com/eclecticiq/cabby :Documentation: http://cabby.readthedocs.org :Information: https://www.eclecticiq.com :Download: https://pypi.python.org/pypi/cabby/ |travis badge| |coveralls.io badge| |docs badge| |requirements badge| .. |travis badge| image:: https://travis-ci.org/eclecticiq/cabby.svg?branch=master :target: https://travis-ci.org/eclecticiq/cabby :alt: Build Status .. |coveralls.io badge| image:: https://coveralls.io/repos/eclecticiq/cabby/badge.svg :target: https://coveralls.io/r/eclecticiq/cabby :alt: Coverage Status .. |docs badge| image:: https://readthedocs.org/projects/cabby/badge/?version=latest :alt: Documentation Status :scale: 100% :target: https://readthedocs.org/projects/cabby/ .. |requirements badge| image:: https://requires.io/github/eclecticiq/cabby/requirements.svg?branch=master :target: https://requires.io/github/eclecticiq/cabby/requirements/?branch=master :alt: Requirements Status A simple Python library for interacting with TAXII servers. Docker -------- To run cabby using docker, execute the following: docker run --rm eclecticiq/cabby taxii-discovery --path https://test.taxiistand.com/read-only/services/discovery Feedback -------- You are encouraged to provide feedback by commenting on open issues or sending us email at cabby@eclecticiq.com cabby-0.1.23/cabby/000077500000000000000000000000001375517671000137515ustar00rootroot00000000000000cabby-0.1.23/cabby/__init__.py000066400000000000000000000036701375517671000160700ustar00rootroot00000000000000''' Cabby, python library for interacting with TAXII servers. ''' from ._version import __version__ # noqa: used in setup.py from .client10 import Client10 from .client11 import Client11 try: from urllib.parse import urlparse except ImportError: # Python 2.7 from urlparse import urlparse def create_client(host=None, port=None, discovery_path=None, use_https=False, discovery_url=None, version="1.1", headers=None): '''Create a client instance (TAXII version specific). ``host``, ``port``, ``use_https``, ``discovery_path`` values can be overridden per request with ``uri`` argument passed to a client's method. :param str host: TAXII server hostname :param int port: TAXII server port :param str discovery_path: Discovery Service relative path :param bool use_https: if HTTPS should be used :param string discovery_url: URL to infer host, port, discovery_path, and use_https. :param string version: TAXII version (1.1 or 1.0) :param dict headers: additional headers to pass with TAXII messages :return: client instance :rtype: :py:class:`cabby.client11.Client11` or :py:class:`cabby.client10.Client10` ''' if discovery_url: parsed = urlparse(discovery_url) if not host and parsed.hostname: host = parsed.hostname if not port and parsed.port: port = parsed.port if not discovery_path and parsed.path: discovery_path = parsed.path if parsed.scheme: use_https = parsed.scheme == 'https' params = dict( host=host, port=port, use_https=use_https, discovery_path=discovery_path, headers=headers) if version == '1.1': return Client11(**params) elif version == '1.0': return Client10(**params) else: raise ValueError("TAXII version %s is not supported" % version) cabby-0.1.23/cabby/_version.py000066400000000000000000000001751375517671000161520ustar00rootroot00000000000000""" Version module. This module defines the package version for use in __init__.py and setup.py. """ __version__ = '0.1.23' cabby-0.1.23/cabby/abstract.py000066400000000000000000000265111375517671000161330ustar00rootroot00000000000000from furl import furl import logging import libtaxii from . import dispatcher, utils from .converters import to_detailed_service_instance_entity from .exceptions import ( AmbiguousServicesError, ClientException, NoURIProvidedError, ServiceNotFoundError, UnsuccessfulStatusError, ) from six.moves import map class AbstractClient(object): ''' Abstract client class. This class can not be used directly, use :py:meth:`cabby.create_client` to create client instances. ''' SUPPORTED_SCHEMES = ['http', 'https'] taxii_version = None def __init__(self, host=None, discovery_path=None, port=None, use_https=False, headers=None, timeout=None): self.host = host self.port = port self.use_https = use_https self.discovery_path = discovery_path self.services = None self.proxies = None self.verify_ssl = True self.ca_cert = None self.cert_file = None self.key_file = None self.key_password = None self.username = None self.password = None self.jwt_url = None self.jwt_token = None self.headers = headers or {} self.timeout = timeout self.log = logging.getLogger( "{}.{}".format(self.__module__, self.__class__.__name__)) def set_auth(self, ca_cert=None, cert_file=None, key_file=None, key_password=None, username=None, password=None, jwt_auth_url=None, verify_ssl=True): ''' Set authentication credentials. ``jwt_auth_url`` is required for JWT based authentication. If it is not specified but ``username`` and ``password`` are provided, client will configure Basic authentication. SSL authentication can be combined with JWT and Basic authentication. :param str ca_cert: a path to CA SSL certificate file :param str cert_file: a path to SSL certificate file :param str key_file: a path to SSL key file :param str username: username, used in basic auth or JWT auth :param str password: password, used in basic auth or JWT auth :param str key_password: same argument as in ``ssl.SSLContext.load_cert_chain`` - may be a function to call to get the password for decrypting the private key or string/bytes/bytearray. It will only be called if the private key is encrypted and a password is necessary. :param str jwt_auth_url: URL used to obtain JWT token :param bool/str verify_ssl: set to False to skip checking host's SSL certificate. Set to True to check certificate against public CAs or set to filepath to check against custom CA bundle. ''' self.ca_cert = ca_cert self.cert_file = cert_file self.key_file = key_file self.key_password = key_password self.username = username self.password = password if jwt_auth_url: self.jwt_url = self._prepare_url(jwt_auth_url) self.verify_ssl = verify_ssl def set_proxies(self, proxies): ''' Set proxy properties. Cause requests to go through a proxy. Must be a dictionary mapping protocol names to URLs of proxies. :param dir proxies: dictionary mapping protocol names to URLs ''' self.proxies = proxies def _prepare_url(self, uri): fu = furl(uri) if fu.scheme and fu.scheme not in self.SUPPORTED_SCHEMES: raise ValueError( 'Scheme "{}" is not supported. Use one of: {}' .format(fu.scheme, ', '.join(self.SUPPORTED_SCHEMES))) use_https = self.use_https or (fu.scheme == 'https') fu.scheme = fu.scheme or ('https' if use_https else 'http') fu.host = fu.host or self.host fu.port = self.port if self.port else ( fu.port or (443 if use_https else 80)) if not fu.host: raise ValueError('Host name is not provided: {}'.format(fu.url)) return fu.url def refresh_jwt_token(self, session=None): ''' Obtain JWT token using provided JWT session, url, username and password. ''' session = session or self.prepare_generic_session() self.jwt_token = dispatcher.obtain_jwt_token( session, self._prepare_url(self.jwt_url), self.username, self.password) session.auth = dispatcher.JWTAuth(self.jwt_token) return self.jwt_token def prepare_generic_session(self): ''' Prepare basic generic session with configured proxies, headers, username/password (if no JWT url configured), cert file, key file and SSL verification flags. ''' return dispatcher.get_generic_session( proxies=self.proxies, headers=self.headers, username=self.username if not self.jwt_url else None, password=self.password if not self.jwt_url else None, cert_file=self.cert_file, key_file=self.key_file, key_password=self.key_password, ca_cert=self.ca_cert, verify_ssl=self.verify_ssl, jwt_token=self.jwt_token, ) def _execute_request(self, request, uri=None, service_type=None): ''' Execute generic TAXII request. A service is defined by ``uri`` parameter or is chosen from pre-cached services by ``service_type``. ''' if not uri and not service_type: raise NoURIProvidedError('URI or service_type needed') elif not uri: service = self._get_service(service_type) uri = service.address if (self.key_file and not self.key_password and utils.if_key_encrypted(self.key_file)): raise ValueError( 'Key file is encrypted but key password was not provided') session = self.prepare_generic_session() uses_jwt = self.jwt_url and self.username and self.password if uses_jwt and not self.jwt_token: self.refresh_jwt_token(session=session) def do_request(): return dispatcher.send_taxii_request( session, self._prepare_url(uri), request, taxii_binding=self.taxii_binding, timeout=self.timeout, ) try: return do_request() except UnsuccessfulStatusError as exc: if uses_jwt and exc.status == libtaxii.ST_UNAUTHORIZED: # An authorization error may indicate JWT token expiry: # transparently try to refresh it, then retry the request. self.refresh_jwt_token(session=session) return do_request() else: raise def _generate_id(self): return libtaxii.common.generate_message_id(version=self.services_version) def _get_service(self, service_type): candidates = self.get_services(service_type=service_type) if not candidates: raise ServiceNotFoundError( "Service with type '{}' is not advertised" .format(service_type)) elif len(candidates) > 1: raise AmbiguousServicesError( "{} services found with type '{}'. Specify the exact URI" .format(len(candidates), service_type)) return candidates[0] def get_services(self, service_type=None, service_types=None): ''' Get services advertised by TAXII server. This method will try to do automatic discovery by calling :py:meth:`discover_services`. :param str service_type: filter services by specific type. Accepted values are listed in :py:data:`cabby.entities.SERVICE_TYPES` :param str service_types: filter services by multiple types. Accepted values are listed in :py:data:`cabby.entities.SERVICE_TYPES` :return: list of service instances :rtype: list of :py:class:`cabby.entities.DetailedServiceInstance` (or :py:class:`cabby.entities.InboxDetailedService`) :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' if self.services: services = self.services else: try: services = self.discover_services() except ClientException as e: self.log.error('Can not autodiscover advertised services') raise e if service_type: return [s for s in services if s.type == service_type] elif service_types: return [s for s in services if s.type in service_types] else: return services def discover_services(self, uri=None, cache=True): ''' Discover services advertised by TAXII server. This method will send discovery request to a service, defined by ``uri`` or constructor's connection parameters. :param str uri: URI path to a specific TAXII service :param bool cache: if discovered services should be cached :return: list of TAXII services :rtype: list of :py:class:`cabby.entities.DetailedServiceInstance` (or :py:class:`cabby.entities.InboxDetailedService`) :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no Discovery servicefound :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' uri = uri or self.discovery_path if not uri: raise NoURIProvidedError('Discovery service URI is not specified') response = self._discovery_request(uri) services = list(map( to_detailed_service_instance_entity, response.service_instances)) self.log.info("%d services discovered", len(services)) if cache: self.services = services return services def __repr__(self): t = '{name}(host={host}, port={port}, discovery_path={discovery_path})' return t.format( name=type(self).__name__, host=self.host, port=self.port, discovery_path=self.discovery_path, ) cabby-0.1.23/cabby/cli/000077500000000000000000000000001375517671000145205ustar00rootroot00000000000000cabby-0.1.23/cabby/cli/__init__.py000066400000000000000000000003621375517671000166320ustar00rootroot00000000000000# flake8: noqa from .discovery import discover_services from .collections import fetch_collections from .poll import poll_content from .push import push_content from .proxy import proxy_content from .subscriptions import manage_subscription cabby-0.1.23/cabby/cli/collections.py000077500000000000000000000013521375517671000174140ustar00rootroot00000000000000 from cabby.constants import ( SVC_COLLECTION_MANAGEMENT, SVC_FEED_MANAGEMENT) from .commons import run_client, get_basic_arg_parser def _runner(client, path, args): collections = [] if path: collections.extend(client.get_collections(uri=path)) else: service_types = [SVC_COLLECTION_MANAGEMENT, SVC_FEED_MANAGEMENT] for service in client.get_services(service_types=service_types): collections.extend( client.get_collections(uri=service.address)) for c in collections: if args.as_xml: print(c.raw.to_xml(pretty_print=True)) else: print(c.raw.to_text()) def fetch_collections(): run_client(get_basic_arg_parser(), _runner) cabby-0.1.23/cabby/cli/commons.py000066400000000000000000000130161375517671000165460ustar00rootroot00000000000000 import logging import sys import argparse from colorlog import ColoredFormatter from .. import create_client logging.captureWarnings(True) log = logging.getLogger(__name__) TAXII_10 = '1.0' TAXII_11 = '1.1' DEFAULT_VERSION = TAXII_11 VERSION_CHOICES = [TAXII_10, TAXII_11] def get_basic_arg_parser(): parser = argparse.ArgumentParser( description="TAXII client", formatter_class=argparse.ArgumentDefaultsHelpFormatter ) parser.add_argument( "--host", dest="host", help="host where the TAXII Service is hosted") parser.add_argument( "--port", dest="port", type=int, help="port where the TAXII Service is hosted") parser.add_argument( "--discovery", dest="discovery", help="path to a TAXII Discovery service") parser.add_argument( "--path", dest="uri", help=("absolute path (as URL) or relative path " "to a specific TAXII Service")) parser.add_argument( "--https", dest="https", action='store_true', help="if the client should use HTTPS") parser.add_argument( "--verify", dest="verify", help=('Set to "no" to skip checking the host\'s SSL certificate.' ' You can also pass the path to a CA file for private certs.' ' Defaults to "yes".')) parser.add_argument( "--timeout", dest="timeout", type=int, help="HTTP request timeout in seconds") parser.add_argument( "--ca-cert", dest="ca_cert", help="CA certificate file") parser.add_argument("--cert", dest="cert", help="certificate file") parser.add_argument("--key", dest="key", help="private key file") parser.add_argument( "--key-password", dest="key_password", help="private key password") parser.add_argument( "--username", dest="username", help="username for authentication") parser.add_argument( "--password", dest="password", help="password for authentication") parser.add_argument( "--jwt-auth", dest="jwt_auth_url", help="JWT authentication URL") parser.add_argument( "--proxy-url", dest="proxy_url", help=("proxy address formatted as URL. Can be set to 'noproxy' " "to force library to not use any proxy")) parser.add_argument( "--proxy-type", dest="proxy_type", choices=['http', 'https'], help="proxy type") parser.add_argument( "--header", dest="headers", action='append', help="header to send with the request, as header:value pair") parser.add_argument( "-v", "--verbose", dest="verbose", action='store_true', help="verbose mode") parser.add_argument( "-x", "--as-xml", dest="as_xml", action='store_true', help="output raw XML") parser.add_argument( "-t", "--taxii-version", dest="version", default=DEFAULT_VERSION, choices=VERSION_CHOICES, help="TAXII version to use") return parser def is_args_valid(args): if not args.uri and not args.discovery: log.error("Either Discovery Service path '--discovery'" " or specific service path '--path' need to be provided") return False return True def prepare_headers(raw_headers): headers = {} for line in raw_headers: header, value = line.split(":", 1) headers[header] = value.strip('"\'') return headers def run_client(parser, run_func): args = parser.parse_args() level = logging.DEBUG if args.verbose else logging.INFO configure_color_logging(level=level) if not is_args_valid(args): sys.exit(1) headers = prepare_headers(args.headers) if args.headers else None client = create_client(host=args.host, discovery_path=args.discovery, port=args.port, use_https=args.https, version=args.version, headers=headers) if args.verify == "yes": verify_ssl = True elif args.verify == "no": verify_ssl = False elif args.verify: verify_ssl = args.verify else: verify_ssl = True client.set_auth( ca_cert=args.ca_cert, cert_file=args.cert, key_file=args.key, username=args.username, password=args.password, key_password=args.key_password, jwt_auth_url=args.jwt_auth_url, verify_ssl=verify_ssl ) client.timeout = args.timeout if args.proxy_url: client.set_proxies({args.proxy_type: args.proxy_url}) try: run_func(client, args.uri, args) except Exception as e: log.error(e, exc_info=args.verbose) def configure_color_logging(level, logger_name=None): logger = logging.getLogger(logger_name) logger.setLevel(level) if level == logging.DEBUG: format_string = ("%(asctime)s %(name)s " "%(log_color)s%(levelname)s:%(reset)s " "%(white)s%(message)s") else: format_string = ("%(asctime)s " "%(log_color)s%(levelname)s:%(reset)s " "%(white)s%(message)s") formatter = ColoredFormatter( format_string, datefmt=None, reset=True, log_colors={ 'DEBUG': 'cyan', 'INFO': 'green', 'WARNING': 'yellow', 'ERROR': 'red', 'CRITICAL': 'red', } ) handlers = logging.StreamHandler(sys.stderr) handlers.setFormatter(formatter) logger.addHandler(handlers) cabby-0.1.23/cabby/cli/discovery.py000077500000000000000000000005741375517671000171120ustar00rootroot00000000000000 from .commons import run_client, get_basic_arg_parser def _runner(client, path, args): services = client.discover_services(uri=path) for s in services: if args.as_xml: print(s.raw.to_xml(pretty_print=True).decode('utf-8')) else: print(s.raw.to_text()) def discover_services(): run_client(get_basic_arg_parser(), _runner) cabby-0.1.23/cabby/cli/poll.py000077500000000000000000000105211375517671000160420ustar00rootroot00000000000000 import os import re import pytz import logging import hashlib import dateutil.parser from .commons import run_client, get_basic_arg_parser log = logging.getLogger(__name__) def extend_arguments(parser): parser.add_argument( "-c", "--collection", dest="collection", help="collection to poll", required=True) parser.add_argument( "--dest-dir", dest="dest_dir", help="directory to save polled content") parser.add_argument( "-l", "--limit", dest="limit", type=int, default=None, help="limit the number of content blocks returned") parser.add_argument( "-r", "--raw", dest="as_raw", action='store_true', help=("output raw TAXII messages " "(use in combination with -x to see them as XML)")) parser.add_argument( "--begin", dest="begin", help="exclusive beginning of time window as ISO8601 formatted date") parser.add_argument( "--end", dest="end", help="inclusive ending of time window as ISO8601 formatted date") parser.add_argument( "-b", "--bindings", dest="bindings", help=("Only return data for the listed content bindings, using " "a string with multiple items seperated by a ','. " "Defauls to all content.")) parser.add_argument( "-s", "--subscription", dest="subscription_id", help="ID of an existing subscription") parser.add_argument( "--count-only", dest="count_only", action='store_true', help="retrieve only count of content blocks") return parser def generate_filename(collection, content_block): collection_name = re.sub(r"[^\w]+", "-", collection) if collection else "" md5 = hashlib.md5() md5.update(content_block.raw.to_xml()) filename = '%s_%s' % (collection_name, md5.hexdigest()) return filename def save_to_dir(dest_dir, collection, content_block, as_raw): filename = generate_filename(collection, content_block) path = os.path.abspath(os.path.join(dest_dir, filename)) with open(path, 'wb') as f: if as_raw: content = content_block.raw.to_xml(pretty_print=True) else: content = content_block.content f.write( content if isinstance(content, bytes) else content.encode('utf-8')) log.info("Content block saved to %s", path) def _runner(client, path, args): if args.limit == 0: return if args.begin: begin = dateutil.parser.parse(args.begin) if not begin.tzinfo: begin = begin.replace(tzinfo=pytz.UTC) else: begin = None if args.end: end = dateutil.parser.parse(args.end) if not end.tzinfo: end = end.replace(tzinfo=pytz.UTC) else: end = None bindings = args.bindings.split(',') if args.bindings else None log.info("Polling using data binding: %s", str(bindings) if bindings else "ALL") if args.count_only: count = client.get_content_count( collection_name=args.collection, begin_date=begin, end_date=end, subscription_id=args.subscription_id, uri=path, content_bindings=bindings, ) if count: log.info("Content blocks count: %s, is partial: %s", count.count, count.is_partial) else: log.warning("Count value was not returned") return blocks = client.poll( collection_name=args.collection, begin_date=begin, end_date=end, subscription_id=args.subscription_id, uri=path, content_bindings=bindings, ) counter = 0 for counter, block in enumerate(blocks, 1): if args.dest_dir: dest_dir = os.path.abspath(args.dest_dir) save_to_dir(dest_dir, args.collection, block, args.as_raw) else: if args.as_raw: if args.as_xml: value = block.raw.to_xml() else: value = block.raw.to_text() else: value = block.content print(value.decode('utf-8')) if args.limit and counter >= args.limit: break log.info("%d blocks polled", counter) def poll_content(): run_client(extend_arguments(get_basic_arg_parser()), _runner) cabby-0.1.23/cabby/cli/proxy.py000077500000000000000000000130741375517671000162630ustar00rootroot00000000000000import argparse import logging import dateutil import pytz from cabby import create_client from cabby.constants import ( CB_STIX_XML_111, CB_CAP_11, CB_SMIME, CB_STIX_XML_10, CB_STIX_XML_101, CB_STIX_XML_11, CB_XENC_122002) from .commons import ( DEFAULT_VERSION, VERSION_CHOICES, configure_color_logging, prepare_headers ) log = logging.getLogger(__name__) # Extra Binding CB_STIX_XML_12 = 'urn:stix.mitre.org:xml:1.2' DEFAULT_BINDING = CB_STIX_XML_111 BINDING_CHOICES = [CB_STIX_XML_111, CB_CAP_11, CB_SMIME, CB_STIX_XML_12, CB_STIX_XML_10, CB_STIX_XML_101, CB_STIX_XML_11, CB_XENC_122002] # Proxy will be: # source polling service => destination inbox service # * Content binding is similar # * Message binding can be different # * collection is different def service_arguments(parser, ident, descriptor): parser.add_argument( "--{}-path".format(ident), dest="{}_path".format(ident), help="Address of the {} service".format(descriptor), required=True) parser.add_argument( "--{}-collection".format(ident), dest="{}_collection".format(ident), help="The collection to {}".format(descriptor), required=True) parser.add_argument( "--{}-taxii-version".format(ident), dest="{}_taxii_version".format(ident), help="The TAXII version to use for {}".format(descriptor), default=DEFAULT_VERSION, choices=VERSION_CHOICES) parser.add_argument( "--{}-username".format(ident), dest="{}_username".format(ident), help="username for {} authentication".format(descriptor)) parser.add_argument( "--{}-password".format(ident), dest="{}_password".format(ident), help="password for {} authentication".format(descriptor)) parser.add_argument( "--{}-jwt-auth".format(ident), dest="{}_jwt_auth_url".format(ident), help="JWT authentication URL for {}".format(descriptor)) parser.add_argument( "--{}-header".format(ident), dest="{}_headers".format(ident), action='append', help="header to send with the {} request, as header:value pair".format( descriptor)) return parser def common_arguments(parser): parser.add_argument( "-v", "--verbose", dest="verbose", action='store_true', help="verbose mode") parser.add_argument( "--binding", dest="binding", default=DEFAULT_BINDING, choices=BINDING_CHOICES, help="content binding") parser.add_argument( "-l", "--limit", dest="limit", type=int, default=None, help="limit the number of content blocks to push") parser.add_argument( "--begin", dest="begin", help="exclusive beginning of time window as ISO8601 formatted date") parser.add_argument( "--end", dest="end", help="inclusive ending of time window as ISO8601 formatted date") return parser def get_blocks(client, args): if args.limit == 0: return if args.begin: begin = dateutil.parser.parse(args.begin) if not begin.tzinfo: begin = begin.replace(tzinfo=pytz.UTC) else: begin = None if args.end: end = dateutil.parser.parse(args.end) if not end.tzinfo: end = end.replace(tzinfo=pytz.UTC) else: end = None blocks = client.poll( collection_name=args.poll_collection, begin_date=begin, end_date=end, uri=args.poll_path, content_bindings=[args.binding], ) counter = 0 for counter, block in enumerate(blocks, 1): yield block.content.decode('utf-8') if args.limit and counter >= args.limit: break log.info("%d blocks polled", counter) def _runner(poll, inbox, args): # Get stuff from poll blocks = get_blocks(poll, args) for block in blocks: log.debug("got block %s", block) inbox.push( block, args.binding, collection_names=[args.inbox_collection], uri=args.inbox_path) log.info("Content block successfully pushed") def get_arg_parser(): parser = argparse.ArgumentParser( description="TAXII Proxy Client", formatter_class=argparse.ArgumentDefaultsHelpFormatter ) # Specific arguments parser = service_arguments(parser, 'inbox', 'inbox') parser = service_arguments(parser, 'poll', 'polling') # Common parser = common_arguments(parser) return parser def run_client(parser, run_func): args = parser.parse_args() level = logging.DEBUG if args.verbose else logging.INFO configure_color_logging(level=level) poll_headers = (prepare_headers(args.poll_headers) if args.poll_headers else None) inbox_headers = (prepare_headers(args.inbox_headers) if args.inbox_headers else None) poll_client = create_client(version=args.poll_taxii_version, headers=poll_headers) inbox_client = create_client(version=args.inbox_taxii_version, headers=inbox_headers) poll_client.set_auth( username=args.poll_username, password=args.poll_password, jwt_auth_url=args.poll_jwt_auth_url ) inbox_client.set_auth( username=args.inbox_username, password=args.inbox_password, jwt_auth_url=args.inbox_jwt_auth_url ) try: run_func(poll_client, inbox_client, args) except Exception as e: log.error(e, exc_info=args.verbose) def proxy_content(): run_client(get_arg_parser(), _runner) cabby-0.1.23/cabby/cli/push.py000077500000000000000000000025211375517671000160540ustar00rootroot00000000000000 import logging from cabby.entities import ContentBinding from cabby.constants import CB_STIX_XML_111 from .commons import run_client, get_basic_arg_parser log = logging.getLogger(__name__) DEFAULT_BINDING = CB_STIX_XML_111 def extend_arguments(parser): parser.add_argument( "-f", "--content-file", dest="content_file", help="file with the content to send", required=True) parser.add_argument( "--binding", dest="binding", default=DEFAULT_BINDING, help="content binding") parser.add_argument( "--subtype", dest="subtypes", action='append', help="the subtype of the content binding") parser.add_argument( "--dest", dest="collections", action='append', help="names of the destination collections") return parser def _runner(client, path, args): with open(args.content_file, 'r') as f: content = f.read() if args.binding and args.subtypes: binding = ContentBinding(args.binding, subtypes=args.subtypes) elif args.binding: binding = args.binding else: binding = None client.push(content, binding, collection_names=args.collections, uri=path) log.info("Content block successfully pushed") def push_content(): run_client(extend_arguments(get_basic_arg_parser()), _runner) cabby-0.1.23/cabby/cli/subscriptions.py000066400000000000000000000043511375517671000200040ustar00rootroot00000000000000from .commons import run_client, get_basic_arg_parser, TAXII_10 ACTIONS = ['subscribe', 'status', 'pause', 'resume', 'unsubscribe'] REQUIRED_SID = ['unsubscribe', 'pause', 'resume'] ONLY_TAXII_11 = ['pause', 'resume'] def extend_arguments(parser): parser.add_argument( "-a", "--action", dest="action", choices=ACTIONS, help="action to perform", required=True) parser.add_argument( "-c", "--collection", dest="collection_name", help="collection name fo subscription", required=True) parser.add_argument( "-s", "--subscription", dest="subscription_id", help="ID of an existing subscription") parser.add_argument( "--count-only", dest="count_only", action='store_true', help="if subscribing only for content counts") return parser def _runner(client, uri, args): if not args.subscription_id and args.action in REQUIRED_SID: raise ValueError("Action '%s' requires subscription ID" % args.action) if args.action in ONLY_TAXII_11 and args.version == TAXII_10: raise ValueError('Action "{}" is not supported in TAXII v{}'.format( args.action, args.version)) if args.action == 'subscribe': response = client.subscribe( args.collection_name, count_only=args.count_only, uri=uri) elif args.action == 'status': response = client.get_subscription_status( args.collection_name, subscription_id=args.subscription_id, uri=uri) elif args.action == 'pause': response = client.pause_subscription( args.collection_name, args.subscription_id, uri=uri) elif args.action == 'resume': response = client.resume_subscription( args.collection_name, args.subscription_id, uri=uri) elif args.action == 'unsubscribe': response = client.unsubscribe( args.collection_name, args.subscription_id, uri=uri) if args.as_xml: print(response.raw.to_xml(pretty_print=True)) else: print(response.raw.to_text()) def manage_subscription(): run_client(extend_arguments(get_basic_arg_parser()), _runner) cabby-0.1.23/cabby/client10.py000066400000000000000000000322661375517671000157530ustar00rootroot00000000000000 import libtaxii.messages_10 as tm10 from . import constants as const from .abstract import AbstractClient from .converters import ( to_subscription_response_entity, to_content_block_entity, to_collection_entities ) from .exceptions import NotSupportedError from .utils import ( pack_content_bindings, get_utc_now, pack_content_binding ) class Client10(AbstractClient): '''Client implementation for TAXII Specification v1.0 Use :py:meth:`cabby.create_client` to create client instances. ''' taxii_binding = const.XML_10_BINDING services_version = const.TAXII_SERVICES_10 def _discovery_request(self, uri): request = tm10.DiscoveryRequest(message_id=self._generate_id()) response = self._execute_request(request, uri=uri) return response def __subscription_status_request(self, action, collection_name, subscription_id=None, uri=None): request_parameters = dict( message_id=self._generate_id(), action=action, feed_name=collection_name, subscription_id=subscription_id ) request = tm10.ManageFeedSubscriptionRequest(**request_parameters) response = self._execute_request( request, uri=uri, service_type=const.SVC_FEED_MANAGEMENT) return to_subscription_response_entity(response, version=10) def get_subscription_status(self, collection_name, subscription_id=None, uri=None): '''Get subscription status from TAXII Feed Management service. Sends a subscription request with action `STATUS`. If no ``subscription_id`` is provided, server will return the list of all available subscriptions for feed with a name specified in ``collection_name``. if ``uri`` is not provided, client will try to discover services and find Feed Management Service among them. :param str collection_name: target feed name :param str subscription_id: subscription ID (optional) :param str uri: URI path to a specific Collection Management service :return: subscription information response :rtype: :py:class:`cabby.entities.SubscriptionResponse` :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' return self.__subscription_status_request( const.ACT_STATUS, collection_name, subscription_id=subscription_id, uri=uri) def unsubscribe(self, collection_name, subscription_id, uri=None): '''Unsubscribe from a subscription. Sends a subscription request with action `UNSUBSCRIBE`. Subscription is identified by ``collection_name`` and ``subscription_id``. if ``uri`` is not provided, client will try to discover services and find Collection Management Service among them. :param str collection_name: target feed name :param str subscription_id: subscription ID :param str uri: URI path to a specific TAXII service :return: subscription information response :rtype: :py:class:`cabby.entities.SubscriptionResponse` :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' return self.__subscription_status_request( const.ACT_UNSUBSCRIBE, collection_name, subscription_id=subscription_id, uri=uri) def subscribe(self, collection_name, inbox_service=None, content_bindings=None, uri=None, count_only=False): '''Create a subscription. Sends a subscription request with action `SUBSCRIBE`. if ``uri`` is not provided, client will try to discover services and find Collection Management Service among them. Content Binding subtypes are not supported in TAXII Specification v1.0. :param str collection_name: target feed name :param `cabby.entities.InboxService` inbox_service: Inbox Service that will accept content pushed by TAXII Server in the context of this subscription :param list content_bindings: a list of strings or :py:class:`cabby.entities.ContentBinding` entities :param str uri: URI path to a specific Collection Management service :param bool count_only: IGNORED. Count Only is not supported in TAXII 1.0 and added here only for method unification purpose. :return: subscription information response :rtype: :py:class:`cabby.entities.SubscriptionResponse` :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' request_parameters = dict( message_id=self._generate_id(), action=const.ACT_SUBSCRIBE, feed_name=collection_name, ) if inbox_service: binding = (inbox_service.message_bindings[0] if inbox_service.message_bindings else '') delivery_parameters = tm10.DeliveryParameters( inbox_protocol=inbox_service.protocol, inbox_address=inbox_service.address, delivery_message_binding=binding ) if content_bindings: delivery_parameters['content_bindings'] = [ tm10.ContentBinding(cb.binding_id) for cb in content_bindings] request_parameters['delivery_parameters'] = delivery_parameters request = tm10.ManageFeedSubscriptionRequest(**request_parameters) response = self._execute_request( request, uri=uri, service_type=const.SVC_FEED_MANAGEMENT) return to_subscription_response_entity(response, version=10) def push(self, content, content_binding, uri=None, timestamp=None): '''Push content into Inbox Service. if ``uri`` is not provided, client will try to discover services and find Inbox Service among them. Content Binding subtypes and Destionation collections are not supported in TAXII Specification v1.0. :param str content: content to push :param content_binding: content binding for a content :type content_binding: string or :py:class:`cabby.entities.ContentBinding` :param datetime timestamp: timestamp label of the content block (current UTC time by default) :param str uri: URI path to a specific Inbox Service :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' content_block = tm10.ContentBlock( content=content, content_binding=pack_content_binding(content_binding, version=10), timestamp_label=timestamp or get_utc_now() ) inbox_message = tm10.InboxMessage(message_id=self._generate_id(), content_blocks=[content_block]) self._execute_request(inbox_message, uri=uri, service_type=const.SVC_INBOX) self.log.debug("Content block successfully pushed") def get_collections(self, uri=None): '''Get collections from Feed Management Service. if ``uri`` is not provided, client will try to discover services and find Feed Management Service among them. :param str uri: URI path to a specific Feed Management service :return: list of collections :rtype: list of :py:class:`cabby.entities.Collection` :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' request = tm10.FeedInformationRequest(message_id=self._generate_id()) response = self._execute_request( request, uri=uri, service_type=const.SVC_FEED_MANAGEMENT) return to_collection_entities(response.feed_informations, version=10) def get_content_count(self, *args, **kwargs): '''Not supported in TAXII 1.0 :raises `cabby.exceptions.NotSupportedError`: not supported in TAXII 1.0 ''' raise NotSupportedError(self.taxii_version) def poll(self, collection_name, begin_date=None, end_date=None, subscription_id=None, content_bindings=None, uri=None): '''Poll content from Polling Service. if ``uri`` is not provided, client will try to discover services and find Polling Service among them. :param str collection_name: feed to poll :param datetime begin_date: ask only for content blocks created after `begin_date` (exclusive) :param datetime end_date: ask only for content blocks created before `end_date` (inclusive) :param str subsctiption_id: ID of the existing subscription :param list content_bindings: list of stings or :py:class:`cabby.entities.ContentBinding` objects :param str uri: URI path to a specific Inbox Service :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' _bindings = pack_content_bindings(content_bindings, version=10) data = dict( message_id=self._generate_id(), feed_name=collection_name, exclusive_begin_timestamp_label=begin_date, inclusive_end_timestamp_label=end_date, content_bindings=_bindings ) if subscription_id: data['subscription_id'] = subscription_id request = tm10.PollRequest(**data) stream = self._execute_request(request, uri=uri, service_type=const.SVC_POLL) for obj in stream: if isinstance(obj, tm10.ContentBlock): yield to_content_block_entity(obj) def fulfilment(self, *args, **kwargs): '''Not supported in TAXII 1.0 :raises `cabby.exceptions.NotSupportedError`: ''' raise NotSupportedError(self.taxii_version) cabby-0.1.23/cabby/client11.py000066400000000000000000000546751375517671000157640ustar00rootroot00000000000000 import libtaxii.messages_11 as tm11 from . import constants as const from .abstract import AbstractClient from .entities import ContentBlockCount from .converters import ( to_subscription_response_entity, to_content_block_entity, to_collection_entities ) from .utils import ( pack_content_bindings, get_utc_now, pack_content_binding ) class Client11(AbstractClient): '''Client implementation for TAXII Specification v1.1 Use :py:meth:`cabby.create_client` to create client instances. ''' taxii_binding = const.XML_11_BINDING services_version = const.TAXII_SERVICES_11 def _discovery_request(self, uri): request = tm11.DiscoveryRequest(message_id=self._generate_id()) response = self._execute_request(request, uri=uri) return response def __subscription_status_request(self, action, collection_name, subscription_id=None, uri=None): request_params = dict( message_id=self._generate_id(), action=action, collection_name=collection_name, subscription_id=subscription_id ) request = tm11.ManageCollectionSubscriptionRequest(**request_params) response = self._execute_request( request, uri=uri, service_type=const.SVC_COLLECTION_MANAGEMENT) return to_subscription_response_entity(response, version=11) def get_subscription_status(self, collection_name, subscription_id=None, uri=None): '''Get subscription status from TAXII Collection Management service. Sends a subscription request with action `STATUS`. If no ``subscription_id`` is provided, server will return the list of all available subscriptions for a collection with a name specified in ``collection_name``. if ``uri`` is not provided, client will try to discover services and find Collection Management Service among them. :param str collection_name: target collection name :param str subscription_id: subscription ID (optional) :param str uri: URI path to a specific Collection Management service :return: subscription information response :rtype: :py:class:`cabby.entities.SubscriptionResponse` :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no Collection Management service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' return self.__subscription_status_request( const.ACT_STATUS, collection_name, subscription_id=subscription_id, uri=uri) def pause_subscription(self, collection_name, subscription_id, uri=None): '''Pause a subscription. Sends a subscription request with action `PAUSE`. Subscription is identified by ``collection_name`` and ``subscription_id``. if ``uri`` is not provided, client will try to discover services and find Collection Management Service among them. :param str collection_name: target collection name :param str subscription_id: subscription ID :param str uri: URI path to a specific Collection Management service :return: subscription information response :rtype: :py:class:`cabby.entities.SubscriptionResponse` :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' return self.__subscription_status_request( const.ACT_PAUSE, collection_name, subscription_id=subscription_id, uri=uri) def resume_subscription(self, collection_name, subscription_id, uri=None): '''Resume a subscription. Sends a subscription request with action `RESUME`. Subscription is identified by ``collection_name`` and ``subscription_id``. if ``uri`` is not provided, client will try to discover services and find Collection Management Service among them. :param str collection_name: target collection name :param str subscription_id: subscription ID :param str uri: URI path to a specific Collection Management service :return: subscription information response :rtype: :py:class:`cabby.entities.SubscriptionResponse` :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' return self.__subscription_status_request( const.ACT_RESUME, collection_name, subscription_id=subscription_id, uri=uri) def unsubscribe(self, collection_name, subscription_id, uri=None): '''Unsubscribe from a subscription. Sends a subscription request with action `UNSUBSCRIBE`. Subscription is identified by ``collection_name`` and ``subscription_id``. if ``uri`` is not provided, client will try to discover services and find Collection Management Service among them. :param str collection_name: target collection name :param str subscription_id: subscription ID :param str uri: URI path to a specific TAXII service :return: subscription information response :rtype: :py:class:`cabby.entities.SubscriptionResponse` :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' return self.__subscription_status_request( const.ACT_UNSUBSCRIBE, collection_name, subscription_id=subscription_id, uri=uri) def subscribe(self, collection_name, count_only=False, inbox_service=None, content_bindings=None, uri=None): '''Create a subscription. Sends a subscription request with action `SUBSCRIBE`. if ``uri`` is not provided, client will try to discover services and find Collection Management Service among them. :param str collection_name: target collection name :param bool count_only: subscribe only to counts and not full content :param `cabby.entities.InboxService` inbox_service: Inbox Service that will accept content pushed by TAXII Server in the context of this subscription :param list content_bindings: a list of strings or :py:class:`cabby.entities.ContentBinding` entities :param str uri: URI path to a specific Collection Management service :return: subscription information response :rtype: :py:class:`cabby.entities.SubscriptionResponse` :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' response_type = const.RT_COUNT_ONLY if count_only else const.RT_FULL sparams = tm11.SubscriptionParameters( response_type=response_type, content_bindings=pack_content_bindings(content_bindings, version=11) ) rparams = dict( message_id=self._generate_id(), action=const.ACT_SUBSCRIBE, collection_name=collection_name, subscription_parameters=sparams, ) if inbox_service: binding = (inbox_service.message_bindings[0] if inbox_service.message_bindings else '') rparams['push_parameters'] = tm11.PushParameters( inbox_protocol=inbox_service.protocol, inbox_address=inbox_service.address, delivery_message_binding=binding ) request = tm11.ManageCollectionSubscriptionRequest(**rparams) response = self._execute_request( request, uri=uri, service_type=const.SVC_COLLECTION_MANAGEMENT) return to_subscription_response_entity(response, version=11) def get_collections(self, uri=None): '''Get collections from Collection Management Service. if ``uri`` is not provided, client will try to discover services and find Collection Management Service among them. :param str uri: URI path to a specific Collection Management service :return: list of collections :rtype: list of :py:class:`cabby.entities.Collection` :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' request = tm11.CollectionInformationRequest( message_id=self._generate_id()) response = self._execute_request( request, uri=uri, service_type=const.SVC_COLLECTION_MANAGEMENT) return to_collection_entities(response.collection_informations, version=11) def push(self, content, content_binding, collection_names=None, timestamp=None, uri=None): '''Push content into Inbox Service. if ``uri`` is not provided, client will try to discover services and find Inbox Service among them. :param str content: content to push :param content_binding: content binding for a content :type content_binding: string or :py:class:`cabby.entities.ContentBinding` :param list collection_names: destination collection names :param datetime timestamp: timestamp label of the content block (current UTC time by default) :param str uri: URI path to a specific Inbox Service :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' content_block = tm11.ContentBlock( content=content, content_binding=pack_content_binding(content_binding, version=11), timestamp_label=timestamp or get_utc_now() ) inbox_message = tm11.InboxMessage(message_id=self._generate_id(), content_blocks=[content_block]) if collection_names: inbox_message.destination_collection_names.extend(collection_names) self._execute_request(inbox_message, uri=uri, service_type=const.SVC_INBOX) self.log.debug("Content block successfully pushed") def _prepare_poll_request(self, collection_name, begin_date=None, end_date=None, subscription_id=None, inbox_service=None, content_bindings=None, count_only=False): data = dict( message_id=self._generate_id(), collection_name=collection_name, exclusive_begin_timestamp_label=begin_date, inclusive_end_timestamp_label=end_date ) if subscription_id: data['subscription_id'] = subscription_id else: _bindings = pack_content_bindings(content_bindings, version=11) poll_params = {'content_bindings': _bindings} if inbox_service: message_bindings = inbox_service.message_bindings[0] \ if inbox_service.message_bindings else [] poll_params['delivery_parameters'] = tm11.DeliveryParameters( inbox_protocol=inbox_service.protocol, inbox_address=inbox_service.address, delivery_message_binding=message_bindings ) poll_params['allow_asynch'] = True if count_only: poll_params['response_type'] = const.RT_COUNT_ONLY else: poll_params['response_type'] = const.RT_FULL data['poll_parameters'] = \ tm11.PollRequest.PollParameters(**poll_params) return tm11.PollRequest(**data) def get_content_count(self, collection_name, begin_date=None, end_date=None, subscription_id=None, inbox_service=None, content_bindings=None, uri=None): '''Get content blocks count for a query. if ``uri`` is not provided, client will try to discover services and find Polling Service among them. If ``subscription_id`` provided, arguments ``content_bindings`` and ``inbox_service`` are ignored. :param str collection_name: collection to poll :param datetime begin_date: ask only for content blocks created after `begin_date` (exclusive) :param datetime end_date: ask only for content blocks created before `end_date` (inclusive) :param str subsctiption_id: ID of the existing subscription :param `cabby.entities.InboxService` inbox_service: Inbox Service that will accept content pushed by TAXII Server in the context of this Poll Request :param list content_bindings: list of stings or :py:class:`cabby.entities.ContentBinding` objects :param str uri: URI path to a specific Inbox Service :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services :rtype: :py:class:`cabby.entities.ContentBlockCount` ''' request = self._prepare_poll_request( collection_name, begin_date=begin_date, end_date=end_date, subscription_id=subscription_id, inbox_service=inbox_service, content_bindings=content_bindings, count_only=True ) response = self._execute_request( request, uri=uri, service_type=const.SVC_POLL) for obj in response: if isinstance(obj, tm11.PollResponse): if obj.record_count: return ContentBlockCount( count=obj.record_count.record_count, is_partial=obj.record_count.partial_count ) else: return None def poll(self, collection_name, begin_date=None, end_date=None, subscription_id=None, inbox_service=None, content_bindings=None, uri=None): '''Poll content from Polling Service. if ``uri`` is not provided, client will try to discover services and find Polling Service among them. If ``subscription_id`` provided, arguments ``content_bindings`` and ``inbox_service`` are ignored. :param str collection_name: collection to poll :param datetime begin_date: ask only for content blocks created after `begin_date` (exclusive) :param datetime end_date: ask only for content blocks created before `end_date` (inclusive) :param str subsctiption_id: ID of the existing subscription :param `cabby.entities.InboxService` inbox_service: Inbox Service that will accept content pushed by TAXII Server in the context of this Poll Request :param list content_bindings: list of stings or :py:class:`cabby.entities.ContentBinding` objects :param str uri: URI path to a specific Inbox Service :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' request = self._prepare_poll_request( collection_name, begin_date=begin_date, end_date=end_date, subscription_id=subscription_id, inbox_service=inbox_service, content_bindings=content_bindings, count_only=False ) stream = self._execute_request(request, uri=uri, service_type=const.SVC_POLL) response = None for obj in stream: if isinstance(obj, tm11.ContentBlock): yield to_content_block_entity(obj) else: response = obj break if response and response.more: part = response.result_part_number while True: part += 1 has_data = False fulfilment_stream = self.fulfilment( collection_name, response.result_id, part_number=part, uri=uri) for block in fulfilment_stream: has_data = True yield block if not has_data: break def fulfilment(self, collection_name, result_id, part_number=1, uri=None): '''Poll content from Polling Service as a part of fulfilment process. if ``uri`` is not provided, client will try to discover services and find Polling Service among them. :param str collection_name: collection to poll :param str result_id: existing polling Result ID :param int part_number: index number of a part from the result set :param str uri: URI path to a specific Inbox Service :raises ValueError: if URI provided is invalid or schema is not supported :raises `cabby.exceptions.HTTPError`: if HTTP error happened :raises `cabby.exceptions.UnsuccessfulStatusError`: if Status Message received and status_type is not `SUCCESS` :raises `cabby.exceptions.ServiceNotFoundError`: if no service found :raises `cabby.exceptions.AmbiguousServicesError`: more than one service with type specified :raises `cabby.exceptions.NoURIProvidedError`: no URI provided and client can't discover services ''' request = tm11.PollFulfillmentRequest( message_id=self._generate_id(), collection_name=collection_name, result_id=result_id, result_part_number=part_number ) stream = self._execute_request(request, uri=uri, service_type=const.SVC_POLL) for obj in stream: if isinstance(obj, tm11.ContentBlock): yield to_content_block_entity(obj) cabby-0.1.23/cabby/constants.py000066400000000000000000000123021375517671000163350ustar00rootroot00000000000000from libtaxii import messages_11 as tm11 from libtaxii import messages_10 as tm10 STREAM_MARKER = 'STREAM' TAXII_11_NS = 'http://taxii.mitre.org/messages/taxii_xml_binding-1.1' TAXII_10_NS = 'http://taxii.mitre.org/messages/taxii_xml_binding-1' XML_11_BINDING = 'urn:taxii.mitre.org:message:xml:1.1' XML_10_BINDING = 'urn:taxii.mitre.org:message:xml:1.0' CERT_EU_JSON_10_BINDING = 'urn:cert.europa.eu:message:json:1.0' VERSIONS = { TAXII_11_NS: XML_11_BINDING, TAXII_10_NS: XML_10_BINDING } MODULES = { TAXII_11_NS: tm11, TAXII_10_NS: tm10 } SUPPORTED_CONTENT_BINDINGS = [ XML_11_BINDING, XML_10_BINDING, CERT_EU_JSON_10_BINDING] BINDINGS_TO_CONTENT_TYPE = { XML_11_BINDING: 'application/xml', XML_10_BINDING: 'application/xml', CERT_EU_JSON_10_BINDING: 'application/json' } TAXII_SERVICES_10 = 'urn:taxii.mitre.org:services:1.0' TAXII_SERVICES_11 = 'urn:taxii.mitre.org:services:1.1' BINDINGS_TO_SERVICES = { XML_10_BINDING: TAXII_SERVICES_10, XML_11_BINDING: TAXII_SERVICES_11, CERT_EU_JSON_10_BINDING: 'urn:taxii.mitre.org:services:1.0' } PROTOCOL_HTTP_10_BINDING = 'urn:taxii.mitre.org:protocol:http:1.0' PROTOCOL_HTTPS_10_BINDING = 'urn:taxii.mitre.org:protocol:https:1.0' SCHEMA_TO_PROTOCOL_BINDINGS = { 'https': PROTOCOL_HTTPS_10_BINDING, 'http': PROTOCOL_HTTP_10_BINDING } # Constant identifying a Status Message MSG_STATUS_MESSAGE = 'Status_Message' # Constant identifying a Discovery Request Message MSG_DISCOVERY_REQUEST = 'Discovery_Request' # Constant identifying a Discovery Response Message MSG_DISCOVERY_RESPONSE = 'Discovery_Response' # Constant identifying a Feed Information Request Message MSG_FEED_INFORMATION_REQUEST = 'Feed_Information_Request' # Constant identifying a Feed Information Response Message MSG_FEED_INFORMATION_RESPONSE = 'Feed_Information_Response' # Constant identifying a Subscription Management Request Message MSG_MANAGE_FEED_SUBSCRIPTION_REQUEST = 'Subscription_Management_Request' # Constant identifying a Subscription Management Response Message MSG_MANAGE_FEED_SUBSCRIPTION_RESPONSE = 'Subscription_Management_Response' # Constant identifying a Poll Request Message MSG_POLL_REQUEST = 'Poll_Request' # Constant identifying a Poll Response Message MSG_POLL_RESPONSE = 'Poll_Response' # Constant identifying a Inbox Message MSG_INBOX_MESSAGE = 'Inbox_Message' # New Message Types in TAXII 1.1 # Constant identifying a Status Message MSG_POLL_FULFILLMENT_REQUEST = 'Poll_Fulfillment' # Constant identifying a Collection Information Request MSG_COLLECTION_INFORMATION_REQUEST = 'Collection_Information_Request' # Constant identifying a Collection Information Response MSG_COLLECTION_INFORMATION_RESPONSE = 'Collection_Information_Response' # Constant identifying a Subscription Request MSG_MANAGE_COLLECTION_SUBSCRIPTION_REQUEST = 'Subscription_Management_Request' # Constant identifying a Subscription Response MSG_MANAGE_COLLECTION_SUBSCRIPTION_RESPONSE = \ 'Subscription_Management_Response' # Constant identifying a Service Type of Inbox SVC_INBOX = 'INBOX' # Constant identifying a Service Type of Poll SVC_POLL = 'POLL' # Constant identifying a Service Type of Feed Management SVC_FEED_MANAGEMENT = 'FEED_MANAGEMENT' # Constant identifying a Service Type of Discovery SVC_DISCOVERY = 'DISCOVERY' # Tuple of all TAXII 1.0 Service Types SVC_TYPES_10 = (SVC_INBOX, SVC_POLL, SVC_FEED_MANAGEMENT, SVC_DISCOVERY) # Renamed Status Types in TAXII 1.1 # Constant identifying a Service Type of Collection Management. # "Feed Management" was renamed to "Collection Management" in TAXII 1.1. SVC_COLLECTION_MANAGEMENT = 'COLLECTION_MANAGEMENT' SVC_TYPES = [ SVC_INBOX, SVC_POLL, SVC_COLLECTION_MANAGEMENT, SVC_DISCOVERY, SVC_FEED_MANAGEMENT] # TAXII 1.0 Action Types # Constant identifying an Action of Subscribe ACT_SUBSCRIBE = 'SUBSCRIBE' # Constant identifying an Action of Unsubscribe ACT_UNSUBSCRIBE = 'UNSUBSCRIBE' # Constant identifying an Action of Status ACT_STATUS = 'STATUS' # Tuple of all TAXII 1.0 Action Types ACT_TYPES_10 = (ACT_SUBSCRIBE, ACT_UNSUBSCRIBE, ACT_STATUS) # Constant identifying an Action of Pause ACT_PAUSE = 'PAUSE' # Constant identifying an Action of Resume ACT_RESUME = 'RESUME' # Subscription Status of Active SS_ACTIVE = 'ACTIVE' # Subscription Status of Paused SS_PAUSED = 'PAUSED' # Subscription Status of Unsubscribed SS_UNSUBSCRIBED = 'UNSUBSCRIBED' # Constant identifying a response type of Full RT_FULL = 'FULL' # Constant identifying a response type of Count only RT_COUNT_ONLY = 'COUNT_ONLY' # Constant identifying a collection type of Data Feed CT_DATA_FEED = 'DATA_FEED' # Constant identifying a collection type of Data Set CT_DATA_SET = 'DATA_SET' # Content Binding ID for STIX XML 1.0 CB_STIX_XML_10 = 'urn:stix.mitre.org:xml:1.0' # Content Binding ID for STIX XML 1.0.1 CB_STIX_XML_101 = 'urn:stix.mitre.org:xml:1.0.1' # Content Binding ID for STIX XML 1.1 CB_STIX_XML_11 = 'urn:stix.mitre.org:xml:1.1' # Content Binding ID for STIX XML 1.1.1 CB_STIX_XML_111 = 'urn:stix.mitre.org:xml:1.1.1' # Content Binding ID for CAP 1.1 CB_CAP_11 = 'urn:oasis:names:tc:emergency:cap:1.1' # Content Binding ID for XML Encryption CB_XENC_122002 = 'http://www.w3.org/2001/04/xmlenc#' # Content Binding ID for SMIME CB_SMIME = 'application/x-pks7-mime' cabby-0.1.23/cabby/converters.py000066400000000000000000000145771375517671000165330ustar00rootroot00000000000000import six from six.moves import map from . import constants as const from .entities import ( ContentBinding, Collection, PushMethod, Subscription, ServiceInstance, InboxDetailedService, DetailedServiceInstance, ContentBlock, InboxService, SubscriptionParameters, SubscriptionResponse) def to_collection_entities(collections, version): return [to_collection_entity(c, version) for c in collections] def to_collection_entity(collection, version): push_methods = [] for pm in collection.push_methods: method = PushMethod( protocol=pm.push_protocol, message_bindings=pm.push_message_bindings ) method.raw = pm push_methods.append(method) subscription_methods = [] for sm in collection.subscription_methods: instance = ServiceInstance( protocol=sm.subscription_protocol, address=sm.subscription_address, message_bindings=sm.subscription_message_bindings ) instance.raw = sm subscription_methods.append(instance) content_bindings = to_content_binding_entities( collection.supported_contents) polling_services = [] for i in collection.polling_service_instances: instance = ServiceInstance( protocol=i.poll_protocol, address=i.poll_address, message_bindings=i.poll_message_bindings ) instance.raw = i polling_services.append(instance) if version == 10: name = collection.feed_name description = collection.feed_description coll_type = Collection.TYPE_FEED volume = None inboxes = None elif version == 11: name = collection.collection_name description = collection.collection_description coll_type = collection.collection_type volume = collection.collection_volume inboxes = [] for inbox in collection.receiving_inbox_services: inboxes.append(InboxService( protocol=inbox.inbox_protocol, address=inbox.inbox_address, message_bindings=inbox.inbox_message_bindings, content_bindings=to_content_binding_entities( inbox.supported_contents) )) collection_entity = Collection( name=name, description=description, type=coll_type, available=collection.available, push_methods=push_methods, subscription_methods=subscription_methods, content_bindings=content_bindings, polling_services=polling_services, volume=volume, receiving_inboxes=inboxes ) collection_entity.raw = collection return collection_entity def to_content_binding_entity(raw_binding): if isinstance(raw_binding, six.string_types): binding = ContentBinding(raw_binding) else: binding = ContentBinding( id=raw_binding.binding_id, subtypes=raw_binding.subtype_ids) binding.raw = raw_binding return binding def to_content_binding_entities(raw_bindings): return list(map(to_content_binding_entity, raw_bindings)) def to_detailed_service_instance_entity(service): params = dict( type=service.service_type, version=service.services_version, protocol=service.protocol_binding, address=service.service_address, message_bindings=service.message_bindings, available=service.available, message=service.message ) if service.service_type == const.SVC_INBOX: cls = InboxDetailedService params['content_bindings'] = to_content_binding_entities( service.inbox_service_accepted_content) else: cls = DetailedServiceInstance instance = cls(**params) instance.raw = service return instance def convert_to_bytes(content): if isinstance(content, six.text_type): return content.encode('utf-8') return content def to_content_block_entity(block): b = ContentBlock( content=convert_to_bytes(block.content), content_binding=to_content_binding_entity(block.content_binding), timestamp=block.timestamp_label, ) b.raw = block return b def to_subscription_entity(subscription, version): params = dict( subscription_id=subscription.subscription_id, ) params['poll_instances'] = [] for i in subscription.poll_instances: instance = ServiceInstance( protocol=i.poll_protocol, address=i.poll_address, message_bindings=i.poll_message_bindings ) instance.raw = i params['poll_instances'].append(instance) raw_delivery_parameters = ( subscription.delivery_parameters if version == 10 else subscription.push_parameters) if raw_delivery_parameters: if version == 10: bindings = to_content_binding_entities( raw_delivery_parameters.content_bindings) else: bindings = None _message_bindings = [raw_delivery_parameters.delivery_message_binding] parameters = InboxService( address=raw_delivery_parameters.inbox_address, protocol=raw_delivery_parameters.inbox_protocol, message_bindings=_message_bindings, content_bindings=bindings ) parameters.raw = raw_delivery_parameters params['delivery_parameters'] = parameters if version == 11: sp = subscription.subscription_parameters parameters = SubscriptionParameters( response_type=sp.response_type, content_bindings=to_content_binding_entities(sp.content_bindings) ) parameters.raw = sp params.update({ 'subscription_parameters': parameters, 'status': subscription.status }) entity = Subscription(**params) entity.raw = subscription return entity def to_subscription_response_entity(raw_response, version): subscriptions = [] for s in raw_response.subscription_instances: subscriptions.append(to_subscription_entity(s, version)) collection_name = (raw_response.collection_name if version == 11 else raw_response.feed_name) response = SubscriptionResponse( collection_name=collection_name, message=raw_response.message, subscriptions=subscriptions ) response.raw = raw_response return response cabby-0.1.23/cabby/dispatcher.py000066400000000000000000000346561375517671000164670ustar00rootroot00000000000000from collections import namedtuple import json import os import ssl import sys import logging from six import StringIO import furl import gzip import requests from lxml import etree from six.moves import urllib from requests.auth import AuthBase, HTTPBasicAuth from libtaxii import messages_11 as tm11 from libtaxii import messages_10 as tm10 from . import constants as const from ._version import __version__ as cabby_version from .exceptions import ( UnsuccessfulStatusError, HTTPError, InvalidResponseError) log = logging.getLogger(__name__) def raise_http_error(status_code, response_stream=None): if log.isEnabledFor(logging.DEBUG) and response_stream: body = response_stream.read() log.debug("Response:\n%s", body.decode('utf-8')) raise HTTPError(status_code) def send_taxii_request( session, url, request, taxii_binding=None, timeout=None): ''' Send XML message to a TAXII service and parse a response. ''' log.info("Sending {} to {}".format(request.message_type, url)) request_body = request.to_xml(pretty_print=True) log.debug("Request:\n%s", request_body.decode('utf-8')) session = get_taxii_session( session, url_scheme=furl.furl(url).scheme, message_binding=taxii_binding) stream, headers = request_stream(session, url, request_body, timeout) gen = _parse_response(stream, headers, version=request.version) obj = next(gen) if obj == const.STREAM_MARKER: return gen elif hasattr(obj, 'status_type'): if obj.status_type != 'SUCCESS': raise UnsuccessfulStatusError(obj) else: return None return obj def request_stream(session, url, request_body, timeout, headers=None): if session._cabby_key_password: # Workaround until # https://github.com/kennethreitz/requests/issues/2519 is fixed try: response = request_with_key_password( session, url, request_body, timeout, headers) except urllib.error.HTTPError as e: log.error( "Error while connecting to {}".format(url), exc_info=True) raise_http_error(e.getcode()) stream, headers = response, response.headers else: response = session.post( url, data=request_body, stream=True, timeout=timeout, headers=headers) if not response.ok: raise_http_error(response.status_code, response.raw) stream, headers = response.raw, response.headers if 'gzip' in headers.get('content-encoding', ''): if sys.version_info < (3, 2): # It is impossible to seek on a HTTP response inside gzipped stream # in gzip lib for python < v3.2, so we are forced to read all # stream content into memory stream = StringIO(stream.read()) stream = gzip.GzipFile(fileobj=stream) return stream, headers def _cleanup_batch(curr_elem, batch): current_parent = curr_elem.getparent() for elem in batch: if elem.getparent() is not None: parent = elem.getparent() else: parent = current_parent parent.remove(elem) del batch[:] def _stream_poll_response(namespace, stream): module = const.MODULES[namespace] response_cls = const.MODULES[namespace].PollResponse batch_max_size = 3 to_delete_batch = [] for action, elem in stream: if action == 'end': tag = elem.xpath('local-name()') # If current element is ContentBlock if tag == module.ContentBlock.NAME: obj = module.ContentBlock.from_etree(elem) # If current element is PollResponse # meaning that this is a last one elif tag == response_cls.message_type: _cleanup_batch(elem, to_delete_batch) obj = response_cls.from_etree(elem) else: continue if log.isEnabledFor(logging.DEBUG): log.debug("Stream element:\n{}" .format(etree.tostring(elem).decode('utf-8'))) yield obj # Cleaning up element to free up memory elem.clear() # Removing all elements from a batch if it is time if len(to_delete_batch) >= batch_max_size: _cleanup_batch(elem, to_delete_batch) # Postponing removal of the element from a tree # to avoid memory corruption (libxml2 crashes) to_delete_batch.append(elem) def _parse_response(stream, headers, version): content_type = headers.get('X-TAXII-Content-Type') if not content_type: body = stream.read() headers = '\n'.join([ '{}={}'.format(k, v) for k, v in headers.items()]) log.debug( "Invalid response:\n{}\n{}".format(headers, body)) raise InvalidResponseError("Invalid response received") elif content_type not in const.SUPPORTED_CONTENT_BINDINGS: raise ValueError('Unsupported X-TAXII-Content-Type: {}' .format(content_type)) elif content_type == const.CERT_EU_JSON_10_BINDING: body = stream.read() yield tm10.get_message_from_json(body) support_huge_trees = not os.environ.get('CABBY_NO_HUGE_TREES') gen = etree.iterparse( stream, events=('start', 'end'), # inject default attributes from DTD or XMLSchema attribute_defaults=False, # validate against a DTD referenced by the document dtd_validation=False, # use DTD for parsing load_dtd=False, # prevent network access for related files (default: True) no_network=True, # try hard to parse through broken XML recover=False, # discard blank text nodes that appear ignorable remove_blank_text=False, # discard comments remove_comments=False, # discard processing instructions remove_pis=False, # replace CDATA sections by normal text content (default: True) strip_cdata=True, # save memory for short text content (default: True) compact=True, # replace entities by their text value (default: True) resolve_entities=False, # enable/disable security restrictions and support very deep # trees and very long text content huge_tree=support_huge_trees) action, root = next(gen) namespace = etree.QName(root).namespace message_type = root.xpath('local-name()') if namespace not in const.VERSIONS: raise ValueError( 'Unsupported namespace: {}'.format(namespace)) elif version != const.VERSIONS[namespace]: raise InvalidResponseError( "Response TAXII version '{}' " "does not match request version '{}'" .format(const.VERSIONS[namespace], version)) if message_type in [tm11.PollResponse.message_type, tm10.PollResponse.message_type]: yield const.STREAM_MARKER for obj in _stream_poll_response(namespace, gen): yield obj else: # Walk tree iterator to the end to fill up root element for _ in gen: pass if log.isEnabledFor(logging.DEBUG): log.debug("Response:\n%s", etree.tostring(root, pretty_print=True).decode('utf-8')) yield _parse_full_tree(content_type, message_type, root) def _parse_full_tree(content_type, message_type, elem): if content_type == const.XML_10_BINDING: tm = tm10 elif content_type == const.XML_11_BINDING: tm = tm11 else: raise ValueError( "Unsupported content type binding {}".format(content_type)) if message_type == const.MSG_DISCOVERY_REQUEST: return tm.DiscoveryRequest.from_etree(elem) elif message_type == const.MSG_DISCOVERY_RESPONSE: return tm.DiscoveryResponse.from_etree(elem) elif message_type == const.MSG_POLL_REQUEST: return tm.PollRequest.from_etree(elem) elif message_type == const.MSG_POLL_RESPONSE: return tm.PollResponse.from_etree(elem) elif message_type == const.MSG_STATUS_MESSAGE: return tm.StatusMessage.from_etree(elem) elif message_type == const.MSG_INBOX_MESSAGE: return tm.InboxMessage.from_etree(elem) if tm == tm11: # TAXII 1.1 specific if message_type == const.MSG_COLLECTION_INFORMATION_REQUEST: return tm.CollectionInformationRequest.from_etree(elem) elif message_type == const.MSG_COLLECTION_INFORMATION_RESPONSE: return tm.CollectionInformationResponse.from_etree(elem) elif message_type == const.MSG_MANAGE_COLLECTION_SUBSCRIPTION_REQUEST: return tm.ManageCollectionSubscriptionRequest.from_etree(elem) elif message_type == const.MSG_MANAGE_COLLECTION_SUBSCRIPTION_RESPONSE: return tm.ManageCollectionSubscriptionResponse.from_etree(elem) elif message_type == const.MSG_POLL_FULFILLMENT_REQUEST: return tm.PollFulfillmentRequest.from_etree(elem) elif tm == tm10: # TAXII 1.0 specific if message_type == const.MSG_FEED_INFORMATION_REQUEST: return tm.FeedInformationRequest.from_etree(elem) elif message_type == const.MSG_FEED_INFORMATION_RESPONSE: return tm.FeedInformationResponse.from_etree(elem) elif message_type == const.MSG_MANAGE_FEED_SUBSCRIPTION_REQUEST: return tm.ManageFeedSubscriptionRequest.from_etree(elem) elif message_type == const.MSG_MANAGE_FEED_SUBSCRIPTION_RESPONSE: return tm.ManageFeedSubscriptionResponse.from_etree(elem) raise ValueError('Unknown message_type: %s' % message_type) class JWTAuth(AuthBase): """Attaches JWT Authentication to the given Request object.""" def __init__(self, token): self.token = token def __call__(self, r): r.headers['Authorization'] = 'Bearer {}'.format(self.token) return r def get_generic_session( proxies=None, headers=None, username=None, password=None, cert_file=None, key_file=None, key_password=None, ca_cert=None, verify_ssl=True, jwt_token=None, ): session = requests.Session() if ca_cert: session.verify = ca_cert else: session.verify = verify_ssl if proxies: session.proxies = proxies if headers: session.headers.update(headers) session.headers['User-Agent'] = 'Cabby {}'.format(cabby_version) if username and password: session.auth = HTTPBasicAuth(username, password) if cert_file and key_file: session.cert = (cert_file, key_file) if jwt_token: session.auth = JWTAuth(jwt_token) session._cabby_key_password = key_password return session def set_jwt_token(session, jwt_token): session.auth = JWTAuth(jwt_token) return session def get_taxii_session(session, url_scheme='https', content_type=None, message_binding=const.XML_11_BINDING, service_binding=None): if not content_type: if message_binding not in const.BINDINGS_TO_CONTENT_TYPE: raise ValueError('No content type provided') content_type = const.BINDINGS_TO_CONTENT_TYPE[message_binding] if not service_binding: if message_binding not in const.BINDINGS_TO_SERVICES: raise ValueError('No service binding provided') service_bindings = const.BINDINGS_TO_SERVICES[message_binding] if url_scheme not in const.SCHEMA_TO_PROTOCOL_BINDINGS: raise ValueError( 'No known protocol bindings for scheme {}'.format(url_scheme)) session.headers.update({ 'Content-Type': content_type, 'Accept': content_type, 'X-TAXII-Content-Type': message_binding, 'X-TAXII-Accept': message_binding, 'X-TAXII-Services': service_bindings, 'X-TAXII-Protocol': const.SCHEMA_TO_PROTOCOL_BINDINGS[url_scheme] }) return session def obtain_jwt_token(session, jwt_url, username, password, timeout=None): log.info("Obtaining JWT token from {}".format(jwt_url)) request_data = json.dumps({'username': username, 'password': password}) request_body = request_data.encode('utf-8') headers = {'Content-Type': 'application/json'} stream, headers = request_stream( session, jwt_url, request_body, timeout, headers) response_body = stream.read().decode('utf-8') if not response_body: raise ValueError("empty response") response_data = json.loads(response_body) if 'token' not in response_data: log.debug("Incorrect JWT response:\n{}".format(response_body)) raise ValueError("No token found in JWT auth response") return response_data['token'] def request_with_key_password( session, url, request_body, timeout=None, headers=None): if sys.version_info < (2, 7, 9): raise ValueError( 'Key password specification is not supported in Python < v2.7.9') if session.auth: # Using Requests Session's auth handlers to fill in proper headers DummyRequest = namedtuple('DummyRequest', ['headers']) request_headers = session.auth( DummyRequest(headers=session.headers)).headers else: request_headers = session.headers if headers: request_headers.update(headers) # Take the TLS details from the session object and use them with urllib. # See also 'get_generic_session' which sets many of these attributes. # session 'verify' attribute can be a bool or a path to a CA bundle: ca_cert = None if not isinstance(session.verify, bool): ca_cert = session.verify context = ssl.create_default_context( ssl.Purpose.CLIENT_AUTH, cafile=ca_cert) cert_file, key_file = session.cert key_password = session._cabby_key_password context.load_cert_chain(cert_file, key_file, password=key_password) if session.verify: context.verify_mode = ssl.CERT_REQUIRED if not ca_cert: context.set_default_verify_paths() else: context.verify_mode = ssl.CERT_NONE handlers = [urllib.request.HTTPSHandler(context=context)] if session.proxies: handlers.append(urllib.request.ProxyHandler(session.proxies)) opener = urllib.request.build_opener(*handlers) request = urllib.request.Request(url, request_body, request_headers) if timeout: return opener.open(request, timeout=timeout) else: return opener.open(request) cabby-0.1.23/cabby/entities.py000066400000000000000000000255411375517671000161560ustar00rootroot00000000000000import logging from . import constants as const log = logging.getLogger(__name__) SERVICE_TYPES = set(const.SVC_TYPES) class Entity(object): '''Generic entity.''' raw = None class ContentBlockCount(Entity): '''Content Block count. Represents an amount of content blocks in a poll query result. :param int count: amount of content blocks in a poll query result. :param bool is_partial: indicates whether the provided Record Count is the exact number of applicable records, or if the provided number is a lower bound and there may be more records than stated. ''' def __init__(self, count, is_partial=False): self.count = count self.is_partial = is_partial def __repr__(self): t = '{cls}(count={count}, is_partial={is_partial})' return t.format(cls=type(self).__name__, **vars(self)) class Collection(Entity): '''Collection entity. Represents TAXII Collection and TAXII Feed objects. :param str name: name of the collection :param str description: description message :param str type: type of a collection. Supported values are :attr:`TYPE_FEED` and :attr:`TYPE_SET` :param bool available: if a collection marked as available :param list push_methods: availavle push methods for a collection, a list of :py:class:`cabby.entities.PushMethods` :param list content_bindings: a list of :py:class:`cabby.entities.ContentBinding` :param list polling_services: a list of :py:class:`cabby.entities.ServiceInstance` :param list subscription_methods: a list of :py:class:`cabby.entities.ServiceInstance` :param list receiving_inboxes: a list of :py:class:`cabby.entities.InboxService` :param int volume: collection's volume ''' TYPE_FEED = const.CT_DATA_FEED TYPE_SET = const.CT_DATA_SET def __init__(self, name, description, type=TYPE_FEED, available=None, push_methods=None, content_bindings=None, polling_services=None, subscription_methods=None, receiving_inboxes=None, volume=None): if type not in [self.TYPE_FEED, self.TYPE_SET]: log.error("Unknown collection type: %s", type) self.name = name self.description = description self.type = type self.available = available self.content_bindings = content_bindings or [] self.push_methods = push_methods or [] self.polling_services = polling_services or [] self.subscription_methods = subscription_methods or [] self.receiving_inboxes = receiving_inboxes or [] self.volume = volume def __repr__(self): t = '{cls}(name={name}, type={type}, available={available})' return t.format(cls=type(self).__name__, **vars(self)) class ContentBinding(Entity): '''Content Binding entity. Represents TAXII Content Binding. :param str id: Content Binding ID :param list subtypes: Content Subtypes IDs ''' def __init__(self, id, subtypes=None): self.id = id self.subtypes = subtypes or [] def __repr__(self): t = '{cls}(id={id}, subtypes={subtypes})' return t.format(cls=type(self).__name__, **vars(self)) class ServiceInstance(Entity): '''Generic TAXII Service entity. :param str protocol: service Protocol Binding value :param str address: service network address :param list message_bindings: service Message Bindings, as list of strings ''' def __init__(self, protocol, address, message_bindings): self.protocol = protocol self.address = address self.message_bindings = message_bindings def __repr__(self): t = '{cls}(protocol={protocol}, address={address})' return t.format(cls=type(self).__name__, **vars(self)) class InboxService(ServiceInstance): '''Inbox Service entity. Represents TAXII Inbox Service. :param str protocol: service Protocol Binding value :param str address: service network address :param list message_bindings: service Message Bindings, as list of strings :param list content_bindings: a list of :py:class:`cabby.entities.ContentBinding` ''' def __init__(self, protocol, address, message_bindings, content_bindings=None): super(InboxService, self).__init__(protocol, address, message_bindings) self.content_bindings = content_bindings or [] class PushMethod(Entity): '''Push Method entity. Represents TAXII Push Method. :param str protocol: service Protocol Binding value :param list message_bindings: service Message Bindings, as list of strings ''' def __init__(self, protocol, message_bindings): self.protocol = protocol self.message_bindings = message_bindings def __repr__(self): t = '{cls}(protocol={protocol})' return t.format(cls=type(self).__name__, **vars(self)) class SubscriptionParameters(Entity): '''Subscription Parameters Entity. Represents TAXII Subscription Parameters. :param str response_type: response type. Supported values are :attr:`TYPE_FULL` and :attr:`TYPE_COUNTS` :param list content_bindings: a list of :py:class:`cabby.entities.ContentBinding` ''' TYPE_FULL = const.RT_FULL TYPE_COUNT = const.RT_COUNT_ONLY def __init__(self, response_type, content_bindings=None): if response_type not in [self.TYPE_FULL, self.TYPE_COUNT]: log.error("Unknown response type: %s", response_type) self.response_type = response_type self.content_bindings = content_bindings or [] def __repr__(self): t = '{cls}(response_type={response_type})' return t.format(cls=type(self).__name__, **vars(self)) class DetailedServiceInstance(Entity): '''Detailed description of a generic TAXII Service instance :param str type: service type. Supported values are in :py:data:`cabby.entities.SERVICE_TYPES` :param str version: service version. Supported values are :attr:`VERSION_10` and :attr:`VERSION_11` :param str protocol: service Protocol Binding value :param str address: service network address :param list message_bindings: service Message Bindings, as list of strings :param bool available: if service is marked as available :param str message: message attached to a service ''' VERSION_10 = const.TAXII_SERVICES_10 VERSION_11 = const.TAXII_SERVICES_11 PROTOCOL_HTTP = const.PROTOCOL_HTTP_10_BINDING PROTOCOL_HTTPS = const.PROTOCOL_HTTPS_10_BINDING def __init__(self, type, version, protocol, address, message_bindings, available=None, message=None): if version not in [self.VERSION_10, self.VERSION_11]: log.error("Unknown service version: %s", version) if protocol not in [self.PROTOCOL_HTTP, self.PROTOCOL_HTTPS]: log.error("Unknown service protocol: %s", protocol) self.type = type self.version = version self.protocol = protocol self.address = address self.message_bindings = message_bindings self.available = available self.message = message def __repr__(self): t = '{cls}(type={type}, address={address})' return t.format(cls=type(self).__name__, **vars(self)) class InboxDetailedService(DetailedServiceInstance): '''Detailed description of TAXII Inbox Service. :param str type: service type. Supported values are in :py:data:`cabby.entities.SERVICE_TYPES` :param str version: service version. Supported values are :attr:`VERSION_10` and :attr:`VERSION_11` :param str protocol: service Protocol Binding value :param str address: service network address :param list message_bindings: service Message Bindings, as list of strings :param list content_bindings: a list of :py:class:`cabby.entities.ContentBinding` :param bool available: if service is marked as available :param str message: message attached to a service ''' def __init__(self, content_bindings, **kwargs): super(InboxDetailedService, self).__init__(**kwargs) self.content_bindings = content_bindings class ContentBlock(Entity): '''Content Block entity. Represents TAXII Content Block. :param str content: TAXII message payload :param `cabby.entities.ContentBinding` content_binding: Content Binding :param datetime timestamp: content block timestamp label ''' def __init__(self, content, content_binding, timestamp): self.content = content self.binding = content_binding self.timestamp = timestamp def __repr__(self): t = '{cls}(timestamp={timestamp})' return t.format(cls=type(self).__name__, **vars(self)) class SubscriptionResponse(Entity): '''Subscription Response entity. :param str collection_name: collection name :param str message: message attached to Subscription Response :param list subscriptions: a list of `cabby.entities.Subscription` ''' def __init__(self, collection_name, message=None, subscriptions=None): self.collection_name = collection_name self.message = message self.subscriptions = subscriptions or [] def __repr__(self): t = '{cls}(collection_name={collection_name})' return t.format(cls=type(self).__name__, **vars(self)) class Subscription(Entity): '''Subscription entity. :param str subscription_id: subscription ID :param str status: subscription status. Supported values are :attr:`STATUS_UNKNOWN`, :attr:`STATUS_ACTIVE`, :attr:`STATUS_PAUSED`, :attr:`STATUS_UNSUBSCRIBED` :param list delivery_parameters: a list of `cabby.entities.InboxService` :param list subscription_parameters: a list of `cabby.entities.SubscriptionParameters` :param list poll_instances: a list of `cabby.entities.ServiceInstance` ''' STATUS_UNKNOWN = 'UNKNOWN' STATUS_ACTIVE = const.SS_ACTIVE STATUS_PAUSED = const.SS_PAUSED STATUS_UNSUBSCRIBED = const.SS_UNSUBSCRIBED def __init__(self, subscription_id, status=STATUS_UNKNOWN, delivery_parameters=None, subscription_parameters=None, poll_instances=None): assert status in [self.STATUS_UNKNOWN, self.STATUS_ACTIVE, self.STATUS_PAUSED, self.STATUS_UNSUBSCRIBED] self.id = subscription_id self.status = status self.delivery_parameters = delivery_parameters self.subscription_parameters = subscription_parameters self.poll_instances = poll_instances def __repr__(self): t = '{cls}(subscription_id={id}, status={status})' return t.format(cls=type(self).__name__, **vars(self)) cabby-0.1.23/cabby/exceptions.py000066400000000000000000000030361375517671000165060ustar00rootroot00000000000000class ClientException(Exception): pass class HTTPError(ClientException): def __init__(self, status_code): super(ClientException, self).__init__( 'HTTP Error: status code {}'.format(status_code)) class InvalidResponseError(ClientException): pass class UnsuccessfulStatusError(ClientException): def __init__(self, taxii_status, *args, **kwargs): msg = "Server Error: {}".format(_status_to_message(taxii_status)) super(UnsuccessfulStatusError, self).__init__(msg, *args, **kwargs) self.status = taxii_status.status_type self.text = taxii_status.to_text() self.raw = taxii_status class AmbiguousServicesError(ClientException): pass class ServiceNotFoundError(ClientException): pass class NoURIProvidedError(ValueError): pass class NotSupportedError(ClientException): def __init__(self, version, *args, **kwargs): super(NotSupportedError, self).__init__( "Not supported in version {}".format(version), *args, **kwargs) def _status_to_message(status): details = [] if status.status_detail: details.append(_dict_to_pairs(status.status_detail)) if status.extended_headers: details.append(_dict_to_pairs(status.extended_headers)) if status.message: details.append(status.message) return "{}: {}".format(status.status_type, "; ".join(details)) def _dict_to_pairs(d): pairs = [] for k, v in list(d.items()): pairs.append('%s=%s' % (k, v)) return ", ".join(pairs) cabby-0.1.23/cabby/utils.py000066400000000000000000000020411375517671000154600ustar00rootroot00000000000000 import pytz from datetime import datetime import libtaxii.messages_11 as tm11 from .entities import ContentBinding def get_utc_now(): return datetime.utcnow().replace(tzinfo=pytz.UTC) def pack_content_binding(content_binding, version): if isinstance(content_binding, ContentBinding): if version == 11: binding = tm11.ContentBinding( binding_id=content_binding.id, subtype_ids=content_binding.subtypes) else: binding = content_binding.id else: if version == 11: binding = tm11.ContentBinding(binding_id=content_binding) else: binding = content_binding return binding def pack_content_bindings(content_bindings, version): if not content_bindings: return None bindings = [] for b in content_bindings: bindings.append(pack_content_binding(b, version)) return bindings def if_key_encrypted(key_file): with open(key_file, 'r') as f: return 'Proc-Type: 4,ENCRYPTED' in f.read() cabby-0.1.23/docker-help.sh000077500000000000000000000011441375517671000154250ustar00rootroot00000000000000#!/bin/sh _cmd="" if [ $0 = '/cabby/docker-help.sh' ]; then _cmd="docker run --rm eclecticiq/cabby " fi cat </dev/null 2>&1; echo $$?), 1) $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) endif # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" @echo " text to make text files" @echo " man to make manual pages" @echo " texinfo to make Texinfo files" @echo " info to make Texinfo files and run them through makeinfo" @echo " gettext to make PO message catalogs" @echo " changes to make an overview of all changed/added/deprecated items" @echo " xml to make Docutils-native XML files" @echo " pseudoxml to make pseudoxml-XML files for display purposes" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: rm -rf $(BUILDDIR)/* html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/Cabby.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/Cabby.qhc" devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/Cabby" @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/Cabby" @echo "# devhelp" epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." $(MAKE) -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." latexpdfja: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through platex and dvipdfmx..." $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." texinfo: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." @echo "Run \`make' in that directory to run these through makeinfo" \ "(use \`make info' here to do that automatically)." info: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo "Running Texinfo files through makeinfo..." make -C $(BUILDDIR)/texinfo info @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." gettext: $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale @echo @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." xml: $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml @echo @echo "Build finished. The XML files are in $(BUILDDIR)/xml." pseudoxml: $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml @echo @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." cabby-0.1.23/docs/_config.yml000066400000000000000000000000331375517671000157440ustar00rootroot00000000000000theme: jekyll-theme-minimalcabby-0.1.23/docs/_templates/000077500000000000000000000000001375517671000157565ustar00rootroot00000000000000cabby-0.1.23/docs/_templates/sidebarintro.html000066400000000000000000000007271375517671000213370ustar00rootroot00000000000000

About Cabby

Cabby is Python TAXII client implementation from EclecticIQ.

Useful Links

cabby-0.1.23/docs/api.rst000066400000000000000000000015351375517671000151300ustar00rootroot00000000000000======================= Cabby API documentation ======================= Module contents =============== .. automodule:: cabby :members: :undoc-members: :show-inheritance: cabby.abstract module --------------------- .. automodule:: cabby.abstract :members: :undoc-members: :show-inheritance: cabby.client10 module --------------------- .. automodule:: cabby.client10 :members: :undoc-members: :show-inheritance: cabby.client11 module --------------------- .. automodule:: cabby.client11 :members: :undoc-members: :show-inheritance: cabby.entities module --------------------- .. automodule:: cabby.entities :members: :undoc-members: :show-inheritance: cabby.exceptions module ----------------------- .. automodule:: cabby.exceptions :members: :undoc-members: :show-inheritance: cabby-0.1.23/docs/changes.rst000066400000000000000000000000341375517671000157600ustar00rootroot00000000000000.. include:: ../CHANGES.rst cabby-0.1.23/docs/conf.py000066400000000000000000000203501375517671000151200ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Cabby documentation build configuration file, created by # sphinx-quickstart on Wed Mar 18 11:09:53 2015. # # This file is execfile()d with the current directory set to its # containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. import sys import os # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. #sys.path.insert(0, os.path.abspath('.')) # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. #needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ 'sphinx.ext.autodoc', ] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8-sig' # The master toctree document. master_doc = 'index' # General information about the project. project = u'Cabby' copyright = u'2016, EclecticIQ' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. exec(open( os.path.join(os.path.dirname(__file__), '../cabby/_version.py')).read()) version = __version__ # The full version, including alpha/beta/rc tags. release = __version__ # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. #language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = ['_build'] # The reST default role (used for this markup: `text`) to use for all # documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # If true, keep warnings as "system message" paragraphs in the built documents. #keep_warnings = False # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. html_theme = 'alabaster' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = None # A shorter title for the navigation bar. Default is the same as html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. #html_logo = None # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. #html_favicon = None # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] # Add any extra paths that contain custom files (such as robots.txt or # .htaccess) here, relative to this directory. These files are copied # directly to the root of the documentation. #html_extra_path = [] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. html_sidebars = { 'index': ['sidebarintro.html', 'sourcelink.html', 'searchbox.html'], '**': ['localtoc.html', 'relations.html', 'sourcelink.html', 'searchbox.html'] } # Additional templates that should be rendered to pages, maps page names to # template names. #html_additional_pages = {} # If false, no module index is generated. #html_domain_indices = True # If false, no index is generated. #html_use_index = True # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. #html_show_sourcelink = True # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. #html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. #html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. #html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = None # Output file base name for HTML help builder. htmlhelp_basename = 'Cabbydoc' # -- Options for LaTeX output --------------------------------------------- latex_elements = { # The paper size ('letterpaper' or 'a4paper'). #'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). #'pointsize': '10pt', # Additional stuff for the LaTeX preamble. #'preamble': '', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ ('index', 'Cabby.tex', u'Cabby Documentation', u'EclecticIQ', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. #latex_use_parts = False # If true, show page references after internal links. #latex_show_pagerefs = False # If true, show URL addresses after external links. #latex_show_urls = False # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_domain_indices = True # -- Options for manual page output --------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ ('index', 'cabby', u'Cabby Documentation', [u'EclecticIQ'], 1) ] # If true, show URL addresses after external links. #man_show_urls = False # -- Options for Texinfo output ------------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ ('index', 'Cabby', u'Cabby Documentation', u'EclecticIQ', 'Cabby', 'One line description of project.', 'Miscellaneous'), ] # Documents to append as an appendix to all manuals. #texinfo_appendices = [] # If false, no module index is generated. #texinfo_domain_indices = True # How to display URL addresses: 'footnote', 'no', or 'inline'. #texinfo_show_urls = 'footnote' # If true, do not generate a @detailmenu in the "Top" node's menu. #texinfo_no_detailmenu = False cabby-0.1.23/docs/developer.rst000066400000000000000000000025631375517671000163460ustar00rootroot00000000000000=========================== Contributing and developing =========================== .. _cabby project page: https://github.com/EclecticIQ/cabby Reporting issues ================ Cabby uses Github's issue tracker. See the `Cabby project page`_ on Github. Obtaining the source code ========================= The Cabby source code can be found on Github. See the `Cabby project page`_ on Github. Layout ====== The Cabby repository has the following layout: * ``docs/`` - used to build the `documentation `_; * ``cabby/`` - source code; * ``tests/`` - tests. Compiling from source ===================== After cloning the Github repo, just run:: $ python setup.py install Running the tests ================= Almost all Cabby code is covered by the unit tests. Cabby uses `py.test `_ and `tox `_ for running tests. Type ``tox -r`` or ``py.test`` to run the unit tests. Generating the documentation ============================ The documentation is written in `ReStructuredText `_ (reST) format and processed using `Sphinx `_. To build HTML documentation, go to ``docs`` and type ``make html``. .. rubric:: Next steps Continue to :doc:`License `. .. vim: set spell spelllang=en: cabby-0.1.23/docs/index.rst000066400000000000000000000027271375517671000154720ustar00rootroot00000000000000===== Cabby ===== Latest stable release is v0.1.18 (:doc:`Changelog `) .. Release v\ |release|. (:doc:`Changelog `) Cabby is Python TAXII client implementation from `EclecticIQ `_. TAXII (Trusted Automated eXchange of Indicator Information) is a collection of specifications defining a set of services and message exchanges used for sharing cyber threat intelligence information between parties. Check `TAXII homepage `_ to get more information. Cabby is designed from the ground up to act as a Python library and as a command line tool, it's key features are: * **Rich feature set**: supports all TAXII services according to TAXII specification (v1.0 and v1.1). * **Version agnostic**: abstracts specific implementation details and returns version agnostic entities. * **Stream parsing**: heavy TAXII Poll Response messages are parsed on the fly, reducing memory footprint and time until first content block is available. .. rubric:: Documentation contents .. toctree:: :maxdepth: 1 installation user api developer changes license .. rubric:: External links * `Online documentation `_ (Read the docs) * `Project page `_ with source code and issue tracker (Github) * `Python Package Index (PyPI) page `_ with released tarballs * `EclecticIQ `_ cabby-0.1.23/docs/installation.rst000066400000000000000000000037061375517671000170620ustar00rootroot00000000000000================== Installation guide ================== .. highlight:: sh Install Python ============== OpenTAXII works with both latest Python version (3.4) and version 2.7. You can install Python with your operating system’s package manager or download it directly `here `_. You can verify that Python is installed by typing ``python`` from your shell; you should see something like:: $ python Python 2.7.8 (default, Oct 19 2014, 16:02:00) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.54)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> Install Cabby ============= To sandbox the project and protect system-wide python it is recommended to install Cabby into a `virtual environment `_ (*virtualenv*): Create a virtual environment named venv:: $ virtualenv venv Where ``venv`` is a directory to place the new environment. Activate this environment:: $ . venv/bin/activate (venv) $ You can now install the latest Cabby release from the `Python Package Index `_ (PyPI) using ``pip``:: (venv) $ pip install cabby .. note:: Since Cabby has `libtaxii `_ as a dependency, the system libraries `libtaxii` requires need to be installed. Check `libtaxii documentation `_ for the details. To install Cabby from source files: download a tarball, unpack it and install it manually with ``python setup.py install``. Versioning ========== Releases of Cabby are given major.minor.revision version numbers, where major and minor correspond to the roadmap EclecticIQ has. The revision number is used to indicate a bug fix only release. .. rubric:: Next steps Continue with the :doc:`User guide ` to see how to use Cabby. .. vim: set spell spelllang=en: cabby-0.1.23/docs/license.rst000066400000000000000000000000341375517671000157720ustar00rootroot00000000000000.. include:: ../LICENSE.rst cabby-0.1.23/docs/user.rst000066400000000000000000000141731375517671000153370ustar00rootroot00000000000000========== User guide ========== This user guide gives an overview of Cabby. It covers: * using Cabby as a library * using Cabby as a command line tool * configuration via environment variables * Docker quickstart guide Note: this document assumes basic familiarity with TAXII specifications. Visit the `TAXII homepage`_ for more information about its features. .. _`TAXII homepage`: https://taxiiproject.github.io/ Using Cabby as a Python library =============================== Below a few examples of how to use the Cabby in your code. We use `test server `_ instance hosted by `TAXIIstand `_ in examples. Create a client:: from cabby import create_client client = create_client( 'test.taxiistand.com', use_https=True, discovery_path='/read-write/services/discovery') Discover advertised services:: services = client.discover_services() for service in services: print('Service type={s.type}, address={s.address}' .format(s=service)) Poll content from a collection:: content_blocks = client.poll(collection_name='all-data') for block in content_blocks: print(block.content) Fetch the collections from Collection Management Serice (or Feed Management Service):: collections = client.get_collections( uri='https://test.taxiistand.com/read-write/services/collection-management') Push content into Inbox Service:: content = 'content-text' binding = 'urn:stix.mitre.org:xml:1.1.1' client.push( content, binding, uri='/read-write/services/inbox/default') To force client to use `TAXII 1.0 `_ specifications, initiate it with a specific ``version`` argument value:: from cabby import create_client client = create_client('open.taxiistand.com', version='1.0') .. note:: Cabby client instances configured for TAXII 1.0 or TAXII 1.1 we will have slightly different method signatures (see :doc:`Cabby API documentation` for details). Authentication methods ---------------------- It is possible to set authentication parameters for TAXII requests:: from cabby import create_client client = create_client( 'secure.taxiiserver.com', discovery_path='/services/discovery') # basic authentication client.set_auth(username='john', password='p4ssw0rd') # or JWT based authentication client.set_auth( username='john', password='p4ssw0rd', jwt_auth_url='/management/auth' ) # or basic authentication with SSL client.set_auth( username='john', password='p4ssw0rd', cert_file='/keys/ssl.cert', key_file='/keys/ssl.key' ) # or only SSL authentication client.set_auth( cert_file='/keys/ssl.cert', key_file='/keys/ssl.key' ) Using Cabby as a command line tool ================================== During installation Cabby adds a family of the command line tools prefixed with ``taxii-`` to your path: .. highlight:: shell Discover services:: (venv) $ taxii-discovery \ --host test.taxiistand.com \ --path /read-only/services/discovery \ --https Fetch the collections list from Collection Management Service:: (venv) $ taxii-collections \ --path https://test.taxiistand.com/read-only/services/collection-management Poll content from a collection (Polling Service will be autodiscovered in advertised services):: (venv) $ $ taxii-poll \ --host test.taxiistand.com \ --https --collection single-binding-slow \ --discovery /read-only/services/discovery Push content into Inbox Service:: (venv) $ taxii-push \ --host test.taxiistand.com \ --https \ --discovery /read-write/services/discovery \ --content-file /intel/stix/stuxnet.stix.xml \ --binding "urn:stix.mitre.org:xml:1.1.1" \ --subtype custom-subtype Create a subscription:: (venv) $ taxii-subscription \ --host test.taxiistand.com \ --https \ --path /read-write/services/collection-management \ --action subscribe \ --collection collection-A Fetch the collections from a service protected by Basic authentication:: (venv) $ taxii-collections \ --path https://test.taxiistand.com/read-write-auth/services/collection-management \ --username test \ --password test Fetch the collections from a service protected by JWT authentication:: (venv) $ taxii-collections \ --host test.taxiistand.com \ --https \ --path /read-write-auth/services/collection-management \ --username guest \ --password guest \ --jwt-auth /management/auth Copy content blocks from one server to another:: (venv) $ taxii-proxy \ --poll-path https://open.taxiistand.com/services/poll \ --poll-collection vxvault \ --inbox-path https://test.taxiistand.com/read-write/services/inbox-stix \ --inbox-collection stix-data \ --binding urn:stix.mitre.org:xml:1.1.1 Use ``--help`` to get more usage details. .. _configuration_via_env_vars: Configuration via environment variables ======================================= * ``CABBY_NO_HUGE_TREES``: by default Cabby enables support for huge trees in `lxml lib `_ (see `lxml manual `_). This disables security restrictions and enables support for very deep trees and very long text content. To disable this, set ``CABBY_NO_HUGE_TREES`` environment variable to any value. Docker Quickstart ================= To ease the threshold for trying out Cabby, it is possible to use the image provided by EclecticIQ: .. code-block:: shell $ docker run --rm cabby bash This will show you some helpful information on what commands are available, and then give you an interactive shell to play around in. .. rubric:: Next steps See :doc:`Cabby API documentation`. .. vim: set spell spelllang=en: cabby-0.1.23/pytest.ini000066400000000000000000000000001375517671000147100ustar00rootroot00000000000000cabby-0.1.23/requirements-dev.txt000066400000000000000000000001451375517671000167310ustar00rootroot00000000000000-r requirements.txt responses pytest>=4.6 pytest-cov pytest-pythonpath zest.releaser flake8 werkzeug cabby-0.1.23/requirements.txt000066400000000000000000000001101375517671000161450ustar00rootroot00000000000000six libtaxii>=1.1.111 colorlog pytz>=2017.2 furl>=0.4.7 requests>=2.7.0 cabby-0.1.23/setup.cfg000066400000000000000000000002411375517671000145070ustar00rootroot00000000000000[metadata] description-file = README.md license_file = LICENSE.rst [flake8] max-line-length = 120 [zest.releaser] python-file-with-version = cabby/_version.py cabby-0.1.23/setup.py000066400000000000000000000032601375517671000144040ustar00rootroot00000000000000import os from setuptools import setup, find_packages __version__ = None exec(open('cabby/_version.py').read()) def here(*path): return os.path.join(os.path.dirname(__file__), *path) def get_file_contents(filename): with open(here(filename)) as fp: return fp.read() # This is a quick and dirty way to include everything from # requirements.txt as package dependencies. install_requires = get_file_contents('requirements.txt').split() setup( name="cabby", description="TAXII client library", long_description=get_file_contents('README.rst'), url="https://github.com/EclecticIQ/cabby/", author="EclecticIQ", author_email="cabby@eclecticiq.com", version=__version__, license="BSD License", packages=find_packages(exclude=['tests']), entry_points={ 'console_scripts': [ 'taxii-proxy=cabby.cli:proxy_content', 'taxii-poll=cabby.cli:poll_content', 'taxii-push=cabby.cli:push_content', 'taxii-discovery=cabby.cli:discover_services', 'taxii-collections=cabby.cli:fetch_collections', 'taxii-subscription=cabby.cli:manage_subscription', ] }, install_requires=install_requires, classifiers=[ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "Intended Audience :: Information Technology", "License :: OSI Approved :: BSD License", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.4", "Topic :: Internet", "Topic :: Security", "Topic :: Software Development :: Libraries :: Python Modules" ] ) cabby-0.1.23/tests/000077500000000000000000000000001375517671000140335ustar00rootroot00000000000000cabby-0.1.23/tests/fixtures10.py000066400000000000000000000161341375517671000164240ustar00rootroot00000000000000# flake8: noqa HOST = 'example.localhost' DISCOVERY_PATH = '/some/discovery/path' DISCOVERY_URI_HTTP = "http://%s%s" % (HOST, DISCOVERY_PATH) DISCOVERY_URI_HTTPS = "https://%s%s" % (HOST, DISCOVERY_PATH) FEED_MANAGEMENT_PATH = '/some/feeds/path' FEED_MANAGEMENT_URI = "http://%s%s" % (HOST, FEED_MANAGEMENT_PATH) INBOX_PATH = '/some/inbox/path' INBOX_URI = "http://%s%s" % (HOST, INBOX_PATH) POLL_PATH = '/some/poll/path' POLL_URI = "http://%s%s" % (HOST, POLL_PATH) POLL_FEED = "some-feed" CONTENT = '_content_' CONTENT_BINDING = '_binding_' CONTENT_BLOCKS = ("Content Block A", "Content Block B") SUBSCRIPTION_ID = 'some-subscription-id-123' DISCOVERY_RESPONSE = ''' urn:taxii.mitre.org:protocol:http:1.0 %(inbox_uri)s TAXII Inbox Service urn:taxii.mitre.org:message:xml:1.1 urn:taxii.mitre.org:protocol:http:1.0 %(discovery_uri)s TAXII Discovery Service urn:taxii.mitre.org:protocol:https:1.0 %(feed_management_uri)s urn:taxii.mitre.org:message:xml:1.1 urn:taxii.mitre.org:protocol:http:1.0 example2.com/taxii-discovery-service Example2 TAXII Discovery Service ''' % dict(inbox_uri=INBOX_URI, discovery_uri=DISCOVERY_URI_HTTP, feed_management_uri=FEED_MANAGEMENT_URI) FEED_MANAGEMENT_RESPONSE = ''' %(feed_name)s urn:stix.mitre.org:xml:1.0 urn:taxii.mitre.org:protocol:https:1.0 %(poll_uri)s urn:taxii.mitre.org:message:xml:1.1 Another feed urn:stix.mitre.org:xml:1.0 urn:taxii.mitre.org:protocol:https:1.0 %(poll_uri)s urn:taxii.mitre.org:message:xml:1.1 ''' % dict(feed_name=POLL_FEED, poll_uri=POLL_URI) POLL_RESPONSE = ''' 2015-01-23T16:26:22.706007+00:00 urn:stix.mitre.org:xml:1.1.1 %(block_1)s 2015-01-22T15:28:49.947928+00:00 urn:stix.mitre.org:xml:1.1.1 %(block_2)s 2015-01-25T15:28:49.947928+00:00 ''' % dict(feed_name=POLL_FEED, block_1=CONTENT_BLOCKS[0], block_2=CONTENT_BLOCKS[1]) SUBSCRIPTION_RESPONSE = ''' urn:taxii.mitre.org:protocol:https:1.0 %(inbox_uri)s urn:taxii.mitre.org:message:xml:1.1 %(content_binding)s urn:taxii.mitre.org:protocol:https:1.0 %(poll_uri)s urn:taxii.mitre.org:message:xml:1.1 ''' % dict(subscription_id=SUBSCRIPTION_ID, poll_uri=POLL_URI, inbox_uri=INBOX_URI, feed_name=POLL_FEED, content_binding=CONTENT_BINDING) INBOX_RESPONSE = ''' ''' STATUS_MESSAGE_UNAUTHORIZED = """\ """ cabby-0.1.23/tests/fixtures11.py000066400000000000000000000234411375517671000164240ustar00rootroot00000000000000# flake8: noqa HOST = 'example.localhost' DISCOVERY_PATH = '/some/discovery/path' DISCOVERY_URI_HTTP = "http://%s%s" % (HOST, DISCOVERY_PATH) DISCOVERY_URI_HTTPS = "https://%s%s" % (HOST, DISCOVERY_PATH) COLLECTION_MANAGEMENT_PATH = '/some/collections/path' COLLECTION_MANAGEMENT_URI = "http://%s%s" % (HOST, COLLECTION_MANAGEMENT_PATH) INBOX_PATH = '/some/inbox/path' INBOX_URI = "http://%s%s" % (HOST, INBOX_PATH) POLL_PATH = '/some/poll/path' POLL_URI = "http://%s%s" % (HOST, POLL_PATH) POLL_COLLECTION = "some-collection" CONTENT = '_content_' CONTENT_BINDING = '_binding_' CONTENT_BLOCKS = ("Content Block A", "Content Block B") SUBSCRIPTION_ID = 'some-subscription-id-123' DISCOVERY_RESPONSE = ''' urn:taxii.mitre.org:protocol:http:1.0 %(inbox_uri)s TAXII Inbox Service urn:taxii.mitre.org:message:xml:1.1 urn:taxii.mitre.org:protocol:http:1.0 %(discovery_uri)s TAXII Discovery Service urn:taxii.mitre.org:protocol:https:1.0 %(collection_management_uri)s urn:taxii.mitre.org:message:xml:1.1 urn:taxii.mitre.org:protocol:http:1.0 example2.com/taxii-discovery-service Example2 TAXII Discovery Service ''' % dict(inbox_uri=INBOX_URI, discovery_uri=DISCOVERY_URI_HTTP, collection_management_uri=COLLECTION_MANAGEMENT_URI) COLLECTION_MANAGEMENT_RESPONSE = ''' Collection %(collection_name)s urn:taxii.mitre.org:protocol:https:1.0 %(poll_uri)s urn:taxii.mitre.org:message:xml:1.1 CollectionB urn:taxii.mitre.org:protocol:https:1.0 %(poll_uri)s urn:taxii.mitre.org:message:xml:1.1 ''' % dict(collection_name=POLL_COLLECTION, poll_uri=POLL_URI) POLL_RESPONSE = ''' 2015-01-22T15:28:49.931734+00:00 2 %(block_1)s 2015-01-22T15:28:49.947928+00:00 %(block_2)s 2015-01-25T15:28:49.947928+00:00 ''' % dict(collection_name=POLL_COLLECTION, block_1=CONTENT_BLOCKS[0], block_2=CONTENT_BLOCKS[1]) POLL_RESPONSE_WITH_MORE_1 = ''' 2015-01-22T15:28:49.931734+00:00 %(block_1)s 2015-01-22T15:28:49.947928+00:00 ''' % dict(collection_name=POLL_COLLECTION, block_1=CONTENT_BLOCKS[0]) POLL_RESPONSE_WITH_MORE_2 = ''' 2015-01-22T15:28:49.931734+00:00 %(block_2)s 2015-01-22T15:28:49.947928+00:00 ''' % dict(collection_name=POLL_COLLECTION, block_2=CONTENT_BLOCKS[1]) SUBSCRIPTION_RESPONSE = ''' Some subscription message %(subscription_id)s FULL urn:taxii.mitre.org:protocol:https:1.0 %(inbox_uri)s urn:taxii.mitre.org:message:xml:1.1 urn:taxii.mitre.org:protocol:https:1.0 %(poll_uri)s urn:taxii.mitre.org:message:xml:1.1 ''' % dict(subscription_id=SUBSCRIPTION_ID, poll_uri=POLL_URI, inbox_uri=INBOX_URI, collection_name=POLL_COLLECTION) INBOX_RESPONSE = ''' ''' STATUS_MESSAGE_UNAUTHORIZED = """\ """ cabby-0.1.23/tests/ssl_test_files/000077500000000000000000000000001375517671000170555ustar00rootroot00000000000000cabby-0.1.23/tests/ssl_test_files/client.key000066400000000000000000000033461375517671000210530ustar00rootroot00000000000000-----BEGIN RSA PRIVATE KEY----- Proc-Type: 4,ENCRYPTED DEK-Info: AES-256-CBC,B67E948DAB86C06E36B8BB6BA3EF27E9 D9JTUYv4cYUCdd9ZIs3yDdiCRr3ogsLWkXmHYiJdHmMGw/FDXSm+EkJKWPVMDDAp hg6AaRVjtLHQwe8lPpD8COizOHgktXiWiZ1vkfz9Pyb4JHmiB6LY5zhnY8pb/2xx 2F8ZI3rbsVkIVUjeqN2PL+I8VM/FFZ2PPp4O57UnY2aDMJlscE0fvGrUrZpwGasz CTr7S58r8w5cWqAR12J3Qw5crUxf/2YvcxJ1xlaLzVMnYUBVVTxrg6S+3jmsBq9F fbz6GgxRSRoltSeV4WmCaeSrG4lwHx+yZLCHjaje5gijMfOuDq5MwVjS9FtiKnxq ZnEDHFYEevBKFXwlPZYGQcDiTBuyet0n8UWpl1mXMc5gFzgEtz+QLWAc6Ws+Q9C5 +hXnbTWY3XRMGVR2qraUa3tTVolUrBdlwvf515oiy/H67OqVhIH5EmCvumA7yvjd PFJdWlMSpLVHGeABYiez+S9n5qk2J5lGW6jjnt0uh+cR9ZllerQbhUgkCWs81xMV e2FbUdRRdcSbc1QNLtEMtw6PaG5Rupcni8hLyZrR9c6RVQSSKDJimGXLS/hHlEn3 /uxbKM4kzFRLZbXiNB4kidyzVMLZePwc4FCtVzEHtmG6nKSydskCn/Q/KTMMTdb/ 6lO33v+sxuLtx7tDmBhR1WqU0/uXEklUu7HBIfEt8dKpe1ZV/ox+LwvUQjdHN01m REMv1s+q+wGRPiKYdukvXyfphi1rXRjAUzTXwD90fGpgCfiVXdcEO/XlP5sKlfdL sbitpCciWcVuJmqKpKH6buYATlm0bOtE6+dWrdjnMSRyGSGjQFT4BZlfuCBP1C12 utl2jPRi44uiIVviezBvi54eplv+2OypP/muVQYebLw19H+fLtqV0m1yVtThP1cT vURavF+ftLui87LCf7282g3y53JP7cU0QIVFruIVAPnw0Yw4rrCEptgN0tSCYv8V mk/ACOAQ2JF69kS8+W+2MYPrwyt/Rwk7K0oWeX/tP8clp9wZ1bizGwkXgQcWiwvX Axj+egWnGplI9EJh893uecFeEtvQf5UBtmVXgzGDcpUbZG7+VBftnEiWiOvPdGjj /pY2FhXsmcDKFGFt3z3To6urMgVgwmq+o/viycs+VN2q45wxIFFEzY8AsIlaPtzP q+LTGY4pyS4Lfyvt5WR+/pMyf494cdfFl6+H2gBgGOZYJzeYGehiryZWPl1CNCmF ATEAjNZ6haEeZrXmy5YdlulBjrFkMpAbznmQdIRlTk8v/2IjpeIYecg+JkSnFM6t r6fxTFFdZ9rlQmR3UNAcCkoQSDkN6MSWITz07ej26x+xlMSRHAVIPAwXiDrrqViR G6cpWYp+MTzNaQvG/Yt2kcqWI7VrkoJ4DjoGlj/PkBCn+i6Q/fsIX19QsKr6wLRO aPbDZN/2vBrjLY1DHmiYbXdVZ+oQhlTTBkGd+t/d335How4AsVPwaiMjrciDJiZz VBs3TbytGfkRzHrIpLxgRQw52OLiPRAdA5J3ReS8pJ5dQgP17+maeC4eap18fYgR KbqfigvYEeOofm1Lhc2rkyZEeXLq9St3Q0CN0+Gdej4G7tQbH0cITXG/2RqHeVaZ -----END RSA PRIVATE KEY----- cabby-0.1.23/tests/ssl_test_files/client.pem000066400000000000000000000025171375517671000210430ustar00rootroot00000000000000-----BEGIN CERTIFICATE----- MIIDvzCCAqcCCQDNW0hGu6KfqDANBgkqhkiG9w0BAQsFADCBnTELMAkGA1UEBhMC TkwxFjAUBgNVBAgMDU5vcnRoLUhvbGxhbmQxEjAQBgNVBAcMCUFtc3RlcmRhbTET MBEGA1UECgwKRWNsZWN0aWNJUTETMBEGA1UECwwKQ2FiYnkgVGVzdDETMBEGA1UE AwwKY2FiYnktdGVzdDEjMCEGCSqGSIb3DQEJARYUY2FiYnlAZWNsZWN0aWNpcS5j b20wHhcNMTgwMzI4MTI1NjUxWhcNMjEwMTE1MTI1NjUxWjCBpDELMAkGA1UEBhMC TkwxFjAUBgNVBAgMDU5vcnRoLUhvbGxhbmQxEjAQBgNVBAcMCUFtc3RlcmRhbTET MBEGA1UECgwKRWNsZWN0aWNJUTETMBEGA1UECwwKQ2FiYnkgVGVzdDEaMBgGA1UE AwwRY2FiYnktdGVzdC1jbGllbnQxIzAhBgkqhkiG9w0BCQEWFGNhYmJ5QGVjbGVj dGljaXEuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAzDmsgKh6 xSoTxyFkjbhQvoYZAQ7oniAN5u87KssY1Qml14b55oz2PO1BdY8DS/TEYJlW3xkB NR10KGlbqnGyR0NnDjjd4E7KF+0ZV6rlF2d50judY/cibHXUp+N/qChJOyEzCKbD uR7hPFsRicVW1LxyM2XW5s1t6ABsCCDsuQfxvTwumSzznzCTSeJaZQMB478R0eQx RaOry28tgOYgSDCEWQjzjPiQIt1cl/CGz3qiLDNURRQIYEHl/3RwD29AWQHuycTN SQXGdsJPA06wv7W3XQV7n+zt8YTA5d4tS9Gt6ZYpby0tovapxFvZSatpLrBl1/mm wzArOXp8LDRmewIDAQABMA0GCSqGSIb3DQEBCwUAA4IBAQCkaNcQHZT/BgIpaVUP qsHyh5RkE/1tQwbFNYHnWrRcWnphcwZyGX7Oj1rpF5e2+YEt3ucpqKGOECfm/QKK H9t5nCYE2KPBIiNhtiPUljAdejP0ysNqkuSLtkDd7sVJUGlldXsJ3pVPC1jKA4/p XlY5FjdVo5QyIPPQ0x/oS8nmxyd/fybKtdO097Gb0m8SrgAHkUHM6osmaij9mSim UPCHClUrP9VuGml4gK0XXfOiil8+DSdmUNX9Mf020lo9JUJFnne3Xku+jWL8RvtC 9Iwx0f4Twi7m45Vsp1jIXH5zQru62TEt8n5KJ6w0EzceyqzKeeeq75VWhodG5HaU wdt7 -----END CERTIFICATE----- cabby-0.1.23/tests/ssl_test_files/client_no_pass.key000066400000000000000000000032131375517671000225660ustar00rootroot00000000000000-----BEGIN RSA PRIVATE KEY----- MIIEogIBAAKCAQEA0fYP7Q2xmfLoVNjtFsOQwTtYzQcvuR/3mxSXWDrDM/FcSgbX spedvf26tzr8PX68YljJN9IwaRVx+20lykH8ZPKminYSht8RdcLUXH+vzpFqGFFd edKsO+//apNw87KV6rYONp+nbE5HJ7GRiAj3Aop5EAtHEjANdJAizyYsMoJiImtP f8doritQ11QSs59mTPsyRuM2ZRvu3CzloBEzx/ClTXSVAMP7jZFanxXcwnrJYSbL /BKxrbPVyzPYBeMjx5V9GUg/YZcDC62C+s0x1CQjacK9meSIx7sXRap3OPFbZ/9c /OWJiTxx6lzywecoELRX/nfDqsr1gn4/v+GvXwIDAQABAoIBAA7fIO2nhKbk37Py 0YhFPeGR9I5BLg4sx3sCkF29e8oYFHNh3LFXr+KfMDR+RxOMAIfuxgHgL3GiBTrL 1ltnJRt5XHZ+On23GyN+M1CB8s/s2Nj0GmzgkTaFn0/LNbrtMVU4o/UWheNUABI9 r5M1H1ncuQp74gLVyH4zH2QQzhydorho/RQ4C8wHIm/yUu2RlrlmoV/CCtuFql7Q EOCwfuf3wom/LUbL2mFNdqYg2LS5ChzFps69zeiO7LF7cppqGAM6hQmtvacmUC7z H1l1Qn6c7uWxI4zezvEF+zEYg2bZhV7Bm2G4k0H5NyC3lP9DD9OjAem9SQaVtO46 b7MLrzECgYEA8ypZBcT5YuKH6v5M/ndzY5tek2PrTuOuzkNY9XC3NgOafjyunbxR o7kn6z6ttpaV8nr43PP5k5FiWyek9FxZp893myEWbxpCZ4s8iKGQs13ICqb3NhAb kmLqEeaHkgqGt2pISOnwKwq323wWhl9iYOlipgpLa2E5aBTGOAAq/esCgYEA3QsP AdwNcVTL9irThaBnVZUw0UViv5oaoYIVJBfkjAZeLSi5Oar1+SGErmD8c/RtrB07 FRABemjRk1Kz1Y1DM4qP6ldWlFaNqvvm+sH8MXtazNeHaFLWUZ5zeh33hV48FUBH 5CG9nXGWSOU/KNYbYk2oFS4fnlVUeXfmsKSBE10CgYBfU46qsFmD5oKaIS9V4sYd ml8tMNKijqeMvOI29gUc67S5IFjkBVuL754yntPC7K3D7Wl6VTrWGvyP9663DS0o mDCvY/1DeOvnY7JLbesoJe+yHVp0m0Pz00sn0VP8cJv3c3b6/prkhMMnDDJYYzRS Aaxmo0qFwgubPemnMomoGQKBgF2OAqoM0vu2oiTsd15FR7cnT+Qi8+qYdNEK15vR KQBC6bU/WTYZL8Zj815C2lbRi2GfdZQylA2VM66hAFBQW7MC4GqU0KY0A/3sZBSZ 6f2fcgzk5AC5ntAoukNjT5H5EoEEFeluhmyO2Ma9kH/eLvTqXUupm+RNxVUr9E2r Mp5ZAoGAJWKK/jR2Bl/f/mNasbmoxs86wwCAHLd8hJ3YMxiN0sRkQfroV/+9En2H LhWqQHIb+lQQzECwnayf4K+SxQUoR5GxOzLe8QIzcsaFfmSjA7pxlIqvzXBVCQKy /kNpbVHGK/SsA0/YDdFA5XA9lO23+dL0lxsoptScrHP0cTKoMtI= -----END RSA PRIVATE KEY----- cabby-0.1.23/tests/ssl_test_files/client_no_pass.pem000066400000000000000000000025231375517671000225620ustar00rootroot00000000000000-----BEGIN CERTIFICATE----- MIIDwTCCAqkCCQDNW0hGu6KfqTANBgkqhkiG9w0BAQsFADCBnTELMAkGA1UEBhMC TkwxFjAUBgNVBAgMDU5vcnRoLUhvbGxhbmQxEjAQBgNVBAcMCUFtc3RlcmRhbTET MBEGA1UECgwKRWNsZWN0aWNJUTETMBEGA1UECwwKQ2FiYnkgVGVzdDETMBEGA1UE AwwKY2FiYnktdGVzdDEjMCEGCSqGSIb3DQEJARYUY2FiYnlAZWNsZWN0aWNpcS5j b20wHhcNMTgwMzI5MDkwODEzWhcNMjEwMTE2MDkwODEzWjCBpjELMAkGA1UEBhMC TkwxFjAUBgNVBAgMDU5vcnRoLUhvbGxhbmQxEjAQBgNVBAcMCUFtc3RlcmRhbTET MBEGA1UECgwKRWNsZWN0aWNJUTEOMAwGA1UECwwFQ2FiYnkxITAfBgNVBAMMGGNh YmJ5LXRlc3Qtbm8tcGFzc3BocmFzZTEjMCEGCSqGSIb3DQEJARYUY2FiYnlAZWNs ZWN0aWNpcS5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDR9g/t DbGZ8uhU2O0Ww5DBO1jNBy+5H/ebFJdYOsMz8VxKBteyl529/bq3Ovw9frxiWMk3 0jBpFXH7bSXKQfxk8qaKdhKG3xF1wtRcf6/OkWoYUV150qw77/9qk3DzspXqtg42 n6dsTkcnsZGICPcCinkQC0cSMA10kCLPJiwygmIia09/x2iuK1DXVBKzn2ZM+zJG 4zZlG+7cLOWgETPH8KVNdJUAw/uNkVqfFdzCeslhJsv8ErGts9XLM9gF4yPHlX0Z SD9hlwMLrYL6zTHUJCNpwr2Z5IjHuxdFqnc48Vtn/1z85YmJPHHqXPLB5ygQtFf+ d8OqyvWCfj+/4a9fAgMBAAEwDQYJKoZIhvcNAQELBQADggEBACh1tE0dIfOF0Mi2 52976pChfkUGNlVmCbkI8UdwcPq9W3a298s8xbg2s+BnxBZhdRgF4xRepq4Db3h9 S7f9iQ0iCHN61yh0UNS/NgsM/ALGhHFCv1sP8ajs3vMDa4P4X5xoBavc+Pf+TKIm CFq/yfXhnMXzfTvLh82zzaE8LM3x+IpthxLkq3F80OXNgyRs5gVFI3poln8TMWiB I1PnrzBRJcHaod67Mly/iHjH0Zwy6jgkEgrbWLmzkmUXU2kEryt0HlaGO4pj5/2A CeZqZxjgzp5NSkdFUEauHYH6bzwhSv+BT0jZOgm5+zt/lOGU8a1Y+BJmc2UhgWQ1 Rhf+7yc= -----END CERTIFICATE----- cabby-0.1.23/tests/ssl_test_files/root_ca.key000066400000000000000000000032131375517671000212140ustar00rootroot00000000000000-----BEGIN RSA PRIVATE KEY----- MIIEowIBAAKCAQEAv0Iz3k0dzt7pNsZJl7uzz15n3CLl8ig2j/lnwJaguK4bTLid XqvvSrCWQo+v7ospCNVMUHN97sqywKFRINPRx/Oit7YXzULTTmP3GdyCsA8XdEtp VxHkr/LdjqnnBsfP5SYem9qvO2VEnpDiHRTBvM/kouHHTNP13PqVdEBBv8RH/Bnm FZplyfl7hgUWnU60wFqixUEF40GhahR8zXHodCYQcJCucIIRMrSooP+UOhzdm4Sv 5XOpz5ChvDU0iLz8DYVZsTgz53uiAmxmF9SerCGaWNuTe1Kz0L/NI3nYhzjyxP8o jve4+ShvVLs1mSS/4jCdrmKvWmnfw2VQPAZwkQIDAQABAoIBAD2wCq6+zMh2mV3z YrUv83UFDBayotwSIFcbqF0zuC8Y6PE78nYgmpPKlQxu793OwLN6qJyiQ5ILitAa t+IQr+8AJBGBB/hKf4EfOQn8V6KVm4kE8bsFqG8ftFpZbox/Ugyk2OaICFDt/s3u CSA8G6jcnO/0RqX/D6PwptfSCeLhyzY8FmvIa8QKSqVyw74T8lANFNri2uBQxMOv oxZ90GABQAkA6lihBCSFUBW28y+joQadtlBgmlhXbhNOMKVmCcYBo+fAHFTr/zKr Ayyzv5RQSb2jMv8vDpAcKZIv44ifwMIu5Se3sNNJa0Vjx1JjwYRVfCW7x/KM/OUy fHNYzuECgYEA6DLwDmAaSTr81fihGgzQCCTAenTfrPckZR/bcjseBfW4HeASl+13 +L0A1wUGrMulXsNKZjlsqWXLIm14jXeB3+uH6fG6jOOG3VoesxJoxJ/8CihjHaz/ NaAkv/k3gsR9WumKNCMkVLcYgO6/9QyDadH8JwCLTmtP1kUqRAVWKXUCgYEA0tz2 DGrKTu8rv+7XiMYsjsRA8AwUDvu3ouNMszO7DxzR+6T/MqnA6YYI1HR6bmNdA1W2 OCiWv8csYuklxpW3O3mn3K5E2fD8irwv4lqfUdLKiYFJiO/AZanNfCtU2ATqUxFz jtPfTVtMYc4rvEvfUiNmkpeO4wA9SVbm589wqy0CgYEA3xCoOzcpqXYolKogojTy 2Q/tdPa4NqLJ7Pf4yjlus00GTnUHh7qSyXtQxj/G1NNij7HO1HuQHwBycCV4bXcE ZogPbrhRfa3KzvUXjvI9gb8r0tpPtZ+PIQYyp9YT9OUsg9zdbw7fhR7MRHiYIiKe UFdxBErqiTBgZzjUzLnPt00CgYA7vEkf/2jMr8jPGNSKdMnOsB5NYr6T+A9v38Cs y6zkfR+nkHiTW6/A+C2sMks01oVVGoUCp8z9C2mxo6cF+Qftm2tIgz4SF8kTbHiM QWv8/CBQkAXK+1Cf/FiwIn5orAvN+JWqEq+h9IlQ891D/KKJXGoEgVcDS+AQ1MOZ TBT5CQKBgE1qh1BUw50sfBqsLRDKrG1VXy6RYba6cJwVErjGLBLnoLNmEBCnFIhk t+1osKVBO7ZYpFjsvF171tVvL2M1DAfum6plYnovTW+ioylGdsVRikNQkqNuWdww PcXa9FdTFh0b6ar2Nv1+s2XVPHvVksBhjWgysn1tR1R/itgtBTQ9 -----END RSA PRIVATE KEY----- cabby-0.1.23/tests/ssl_test_files/root_ca.pem000066400000000000000000000027001375517671000212050ustar00rootroot00000000000000-----BEGIN CERTIFICATE----- MIIEEjCCAvqgAwIBAgIJAMSDzbM9xjhHMA0GCSqGSIb3DQEBCwUAMIGdMQswCQYD VQQGEwJOTDEWMBQGA1UECAwNTm9ydGgtSG9sbGFuZDESMBAGA1UEBwwJQW1zdGVy ZGFtMRMwEQYDVQQKDApFY2xlY3RpY0lRMRMwEQYDVQQLDApDYWJieSBUZXN0MRMw EQYDVQQDDApjYWJieS10ZXN0MSMwIQYJKoZIhvcNAQkBFhRjYWJieUBlY2xlY3Rp Y2lxLmNvbTAeFw0xODAzMjgxMjUzNTBaFw0yMTAxMTUxMjUzNTBaMIGdMQswCQYD VQQGEwJOTDEWMBQGA1UECAwNTm9ydGgtSG9sbGFuZDESMBAGA1UEBwwJQW1zdGVy ZGFtMRMwEQYDVQQKDApFY2xlY3RpY0lRMRMwEQYDVQQLDApDYWJieSBUZXN0MRMw EQYDVQQDDApjYWJieS10ZXN0MSMwIQYJKoZIhvcNAQkBFhRjYWJieUBlY2xlY3Rp Y2lxLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL9CM95NHc7e 6TbGSZe7s89eZ9wi5fIoNo/5Z8CWoLiuG0y4nV6r70qwlkKPr+6LKQjVTFBzfe7K ssChUSDT0cfzore2F81C005j9xncgrAPF3RLaVcR5K/y3Y6p5wbHz+UmHpvarztl RJ6Q4h0UwbzP5KLhx0zT9dz6lXRAQb/ER/wZ5hWaZcn5e4YFFp1OtMBaosVBBeNB oWoUfM1x6HQmEHCQrnCCETK0qKD/lDoc3ZuEr+Vzqc+Qobw1NIi8/A2FWbE4M+d7 ogJsZhfUnqwhmljbk3tSs9C/zSN52Ic48sT/KI73uPkob1S7NZkkv+Iwna5ir1pp 38NlUDwGcJECAwEAAaNTMFEwHQYDVR0OBBYEFJLjDK/tYibETbI4wZPcilqVmvZ1 MB8GA1UdIwQYMBaAFJLjDK/tYibETbI4wZPcilqVmvZ1MA8GA1UdEwEB/wQFMAMB Af8wDQYJKoZIhvcNAQELBQADggEBAJg7OTXhEmmbCbYEuZJmvwYXQFysSPfFQSG3 v5o1/vB50hUY/8ghukuCuo960tRfJxvw2VdefwuXK9O2LmdqwRmUxeWlsmWlIsL3 CYukUtsWvjk4Nwr119kKSqBCa5pd7dSU2S25DxHhVcYlDp7BvTx3ix0miD+buhOx T+fgwvA18Y+M4JyELNLLV2fB+vNqjZnbGyDw/nXBYfu2CgHWxzlpjHw+8LEIwZx5 c/dyqMqojmISVnYE+Fya4aiVwKLkky38ox6CXfL5smSHRbgqbRdWqQT+ZAnscsWh FO9naXA5aIBWY800uX9iErke729QkegJv15lS7mOHmeTPWoGUtk= -----END CERTIFICATE----- cabby-0.1.23/tests/test_client.py000066400000000000000000000004661375517671000167300ustar00rootroot00000000000000import cabby def test_url_parse(): url = 'https://eiq-test.com:1337/path/to/discovery' client = cabby.create_client(discovery_url=url) assert client.host == 'eiq-test.com' assert client.port == 1337 assert client.use_https is True assert client.discovery_path == '/path/to/discovery' cabby-0.1.23/tests/test_client10.py000066400000000000000000000203421375517671000170640ustar00rootroot00000000000000 import pytest import responses from libtaxii import messages_10 as tm10 from cabby import create_client from cabby import exceptions as exc from cabby import entities from cabby.constants import ( XML_10_BINDING, SVC_INBOX, SVC_DISCOVERY, ) from fixtures10 import ( HOST, POLL_RESPONSE, INBOX_RESPONSE, DISCOVERY_RESPONSE, FEED_MANAGEMENT_RESPONSE, SUBSCRIPTION_RESPONSE, CONTENT_BLOCKS, SUBSCRIPTION_ID, FEED_MANAGEMENT_PATH, POLL_FEED, CONTENT, DISCOVERY_URI_HTTP, DISCOVERY_URI_HTTPS, FEED_MANAGEMENT_URI, INBOX_URI, POLL_URI, POLL_PATH, DISCOVERY_PATH, CONTENT_BINDING) # Utils def create_client_10(**kwargs): client = create_client(HOST, version="1.0", **kwargs) return client def register_uri(uri, body, **kwargs): responses.add( method=responses.POST, url=uri, body=body, content_type='application/xml', stream=True, adding_headers={'X-TAXII-Content-Type': XML_10_BINDING}, **kwargs) def get_sent_message(): body = responses.calls[-1].request.body print(repr(body)) return tm10.get_message_from_xml(body) # Tests def test_no_discovery_path(): client = create_client_10() with pytest.raises(exc.NoURIProvidedError): client.discover_services() def test_no_discovery_path_when_pushing(): client = create_client_10() with pytest.raises(exc.NoURIProvidedError): client.push(CONTENT, CONTENT_BINDING) @responses.activate def test_incorrect_path(): responses.add(responses.POST, DISCOVERY_URI_HTTP, status=404) client = create_client_10(discovery_path=DISCOVERY_PATH) with pytest.raises(exc.HTTPError): client.discover_services() @responses.activate def test_discovery(): register_uri(DISCOVERY_URI_HTTP, DISCOVERY_RESPONSE) client = create_client_10(discovery_path=DISCOVERY_PATH) services = client.discover_services() assert len(services) == 4 assert len([s for s in services if s.type == SVC_INBOX]) == 1 assert len([s for s in services if s.type == SVC_DISCOVERY]) == 2 message = get_sent_message() assert type(message) == tm10.DiscoveryRequest @responses.activate def test_discovery_https(): register_uri(DISCOVERY_URI_HTTPS, DISCOVERY_RESPONSE) client = create_client_10(discovery_path=DISCOVERY_PATH, use_https=True) services = client.discover_services() assert len(services) == 4 message = get_sent_message() assert type(message) == tm10.DiscoveryRequest @responses.activate def test_collections(): register_uri(FEED_MANAGEMENT_URI, FEED_MANAGEMENT_RESPONSE) client = create_client_10() collections = client.get_collections(uri=FEED_MANAGEMENT_PATH) assert len(collections) == 2 assert all(c.type == entities.Collection.TYPE_FEED for c in collections) feed = collections[0] assert len(feed.polling_services) == 1 service = feed.polling_services[0] assert service.address == POLL_URI assert service.protocol is not None assert len(service.message_bindings) == 1 message = get_sent_message() assert type(message) == tm10.FeedInformationRequest @responses.activate def test_collections_with_automatic_discovery(): register_uri(DISCOVERY_URI_HTTP, DISCOVERY_RESPONSE) register_uri(FEED_MANAGEMENT_URI, FEED_MANAGEMENT_RESPONSE) client = create_client_10(discovery_path=DISCOVERY_URI_HTTP) collections = client.get_collections() assert len(collections) == 2 message = get_sent_message() assert type(message) == tm10.FeedInformationRequest @responses.activate def test_poll(): register_uri(POLL_URI, POLL_RESPONSE) client = create_client_10() blocks = list(client.poll(POLL_FEED, uri=POLL_PATH)) assert len(blocks) == 2 message = get_sent_message() assert type(message) == tm10.PollRequest assert message.feed_name == POLL_FEED @responses.activate def test_poll_count_only(): register_uri(POLL_URI, POLL_RESPONSE) client = create_client_10() with pytest.raises(exc.NotSupportedError): client.get_content_count(POLL_FEED, uri=POLL_PATH) @responses.activate def test_poll_with_subscription(): register_uri(POLL_URI, POLL_RESPONSE) client = create_client_10() blocks = list(client.poll(POLL_FEED, subscription_id=SUBSCRIPTION_ID, uri=POLL_PATH)) assert len(blocks) == 2 message = get_sent_message() assert type(message) == tm10.PollRequest assert message.feed_name == POLL_FEED assert message.subscription_id == SUBSCRIPTION_ID @responses.activate def test_poll_with_content_bindings(): register_uri(POLL_URI, POLL_RESPONSE) client = create_client_10() gen = client.poll( POLL_FEED, uri=POLL_PATH, content_bindings=[CONTENT_BINDING]) block_1 = next(gen) print(gen, block_1.content, CONTENT_BLOCKS) assert block_1.content.decode('utf-8') == CONTENT_BLOCKS[0] message = get_sent_message() assert type(message) == tm10.PollRequest assert message.feed_name == POLL_FEED assert len(message.content_bindings) == 1 assert message.content_bindings[0] == CONTENT_BINDING binding = entities.ContentBinding(CONTENT_BINDING, subtypes=['substype-a']) gen = client.poll( POLL_FEED, uri=POLL_PATH, content_bindings=[binding]) block_1 = next(gen) assert block_1.content.decode('utf-8') == CONTENT_BLOCKS[0] message = get_sent_message() assert type(message) == tm10.PollRequest assert len(message.content_bindings) == 1 assert message.content_bindings[0] == binding.id @responses.activate def test_subscribe(): register_uri(FEED_MANAGEMENT_URI, SUBSCRIPTION_RESPONSE) client = create_client_10() response = client.subscribe(POLL_FEED, uri=FEED_MANAGEMENT_PATH) assert response.collection_name == POLL_FEED assert len(response.subscriptions) == 1 subscription = response.subscriptions[0] assert subscription.status == subscription.STATUS_UNKNOWN message = get_sent_message() assert type(message) == tm10.ManageFeedSubscriptionRequest assert message.feed_name == POLL_FEED assert message.action == tm10.ACT_SUBSCRIBE @responses.activate def test_subscribe_with_push(): register_uri(DISCOVERY_URI_HTTP, DISCOVERY_RESPONSE) register_uri(FEED_MANAGEMENT_URI, SUBSCRIPTION_RESPONSE) client = create_client_10(discovery_path=DISCOVERY_PATH) services = client.discover_services() inbox = next((s for s in services if s.type == SVC_INBOX), None) response = client.subscribe(POLL_FEED, inbox_service=inbox, uri=FEED_MANAGEMENT_PATH) assert response.collection_name == POLL_FEED assert len(response.subscriptions) == 1 subscription = response.subscriptions[0] # TAXII 1.0 reply lacks 'status' field assert subscription.status == subscription.STATUS_UNKNOWN message = get_sent_message() assert type(message) == tm10.ManageFeedSubscriptionRequest assert message.feed_name == POLL_FEED assert message.delivery_parameters.inbox_address == inbox.address assert message.action == tm10.ACT_SUBSCRIBE @responses.activate def test_unsubscribe(): register_uri(FEED_MANAGEMENT_URI, SUBSCRIPTION_RESPONSE) client = create_client_10() response = client.unsubscribe(POLL_FEED, SUBSCRIPTION_ID, uri=FEED_MANAGEMENT_PATH) assert response.collection_name == POLL_FEED assert len(response.subscriptions) == 1 # TAXII 1.0 does not reply with 'status' field configured assert response.subscriptions[0].status == \ entities.Subscription.STATUS_UNKNOWN message = get_sent_message() assert type(message) == tm10.ManageFeedSubscriptionRequest assert message.feed_name == POLL_FEED assert message.action == tm10.ACT_UNSUBSCRIBE @responses.activate def test_push(): register_uri(INBOX_URI, INBOX_RESPONSE) client = create_client_10() client.push(CONTENT, CONTENT_BINDING, uri=INBOX_URI) message = get_sent_message() assert type(message) == tm10.InboxMessage assert len(message.content_blocks) == 1 assert message.content_blocks[0].content == CONTENT assert message.content_blocks[0].content_binding == CONTENT_BINDING cabby-0.1.23/tests/test_client11.py000066400000000000000000000312011375517671000170610ustar00rootroot00000000000000 import pytest import responses from libtaxii import messages_11 as tm11 from cabby import create_client from cabby import exceptions as exc from cabby import entities from cabby.constants import ( XML_11_BINDING, SVC_INBOX, SVC_DISCOVERY, RT_COUNT_ONLY ) from fixtures11 import ( HOST, CONTENT_BINDING, POLL_RESPONSE, POLL_RESPONSE_WITH_MORE_1, POLL_RESPONSE_WITH_MORE_2, INBOX_RESPONSE, SUBSCRIPTION_ID, COLLECTION_MANAGEMENT_RESPONSE, POLL_PATH, COLLECTION_MANAGEMENT_PATH, DISCOVERY_RESPONSE, SUBSCRIPTION_RESPONSE, DISCOVERY_PATH, CONTENT_BLOCKS, CONTENT, COLLECTION_MANAGEMENT_URI, DISCOVERY_URI_HTTP, DISCOVERY_URI_HTTPS, INBOX_URI, POLL_URI, POLL_COLLECTION) # Utils def create_client_11(**kwargs): client = create_client(HOST, version="1.1", **kwargs) return client def register_uri(uri, body, **kwargs): responses.add( method=responses.POST, url=uri, body=body, content_type='application/xml', stream=True, adding_headers={'X-TAXII-Content-Type': XML_11_BINDING}, **kwargs) def get_sent_message(): body = responses.calls[-1].request.body print(repr(body)) return tm11.get_message_from_xml(body) # Tests def test_no_discovery_path(): client = create_client_11() with pytest.raises(exc.NoURIProvidedError): client.discover_services() def test_no_discovery_path_when_pushing(): client = create_client_11() with pytest.raises(exc.NoURIProvidedError): client.push(CONTENT, CONTENT_BINDING) @responses.activate def test_incorrect_path(): responses.add(responses.POST, DISCOVERY_URI_HTTP, status=404) client = create_client_11(discovery_path=DISCOVERY_PATH) with pytest.raises(exc.HTTPError): client.discover_services() @responses.activate def test_discovery(): register_uri(DISCOVERY_URI_HTTP, DISCOVERY_RESPONSE) client = create_client_11(discovery_path=DISCOVERY_PATH) services = client.discover_services() assert len(services) == 4 assert all([ isinstance(s, entities.DetailedServiceInstance) for s in services ]) assert len([s for s in services if s.type == SVC_INBOX]) == 1 assert len([s for s in services if s.type == SVC_DISCOVERY]) == 2 message = get_sent_message() assert type(message) == tm11.DiscoveryRequest @responses.activate def test_discovery_https(): register_uri(DISCOVERY_URI_HTTPS, DISCOVERY_RESPONSE) client = create_client_11(discovery_path=DISCOVERY_PATH, use_https=True) services = client.discover_services() assert len(services) == 4 message = get_sent_message() assert type(message) == tm11.DiscoveryRequest @responses.activate def test_collections(): register_uri(COLLECTION_MANAGEMENT_URI, COLLECTION_MANAGEMENT_RESPONSE) client = create_client_11() collections = client.get_collections(uri=COLLECTION_MANAGEMENT_PATH) assert len(collections) == 2 assert all([c.type == entities.Collection.TYPE_FEED for c in collections]) message = get_sent_message() assert type(message) == tm11.CollectionInformationRequest @responses.activate def test_collections_with_automatic_discovery(): register_uri(DISCOVERY_URI_HTTP, DISCOVERY_RESPONSE) register_uri(COLLECTION_MANAGEMENT_URI, COLLECTION_MANAGEMENT_RESPONSE) client = create_client_11(discovery_path=DISCOVERY_URI_HTTP) response = client.get_collections() assert len(response) == 2 message = get_sent_message() assert type(message) == tm11.CollectionInformationRequest @responses.activate def test_poll(): register_uri(POLL_URI, POLL_RESPONSE) client = create_client_11() blocks = list(client.poll(POLL_COLLECTION, uri=POLL_PATH)) assert len(blocks) == 2 message = get_sent_message() assert type(message) == tm11.PollRequest assert message.collection_name == POLL_COLLECTION @responses.activate def test_poll_count_only(): register_uri(POLL_URI, POLL_RESPONSE) client = create_client_11() count = client.get_content_count(POLL_COLLECTION, uri=POLL_PATH) assert count.count == 2 assert not count.is_partial message = get_sent_message() assert type(message) == tm11.PollRequest assert message.collection_name == POLL_COLLECTION assert message.poll_parameters.response_type == RT_COUNT_ONLY @responses.activate def test_poll_with_subscription(): register_uri(POLL_URI, POLL_RESPONSE) client = create_client_11() blocks = list(client.poll(POLL_COLLECTION, subscription_id=SUBSCRIPTION_ID, uri=POLL_PATH)) assert len(blocks) == 2 message = get_sent_message() assert type(message) == tm11.PollRequest assert message.collection_name == POLL_COLLECTION assert message.subscription_id == SUBSCRIPTION_ID @responses.activate def test_poll_with_delivery(): register_uri(DISCOVERY_URI_HTTP, DISCOVERY_RESPONSE) register_uri(POLL_URI, POLL_RESPONSE) client = create_client_11(discovery_path=DISCOVERY_PATH) services = client.discover_services() inbox = next((s for s in services if s.type == SVC_INBOX), None) blocks = list(client.poll(POLL_COLLECTION, inbox_service=inbox, uri=POLL_PATH)) assert len(blocks) == 2 message = get_sent_message() assert type(message) == tm11.PollRequest assert message.collection_name == POLL_COLLECTION returned_addr = message.poll_parameters.delivery_parameters.inbox_address assert returned_addr == inbox.address assert message.poll_parameters.allow_asynch is True @responses.activate def test_poll_with_fullfilment(): register_uri(POLL_URI, POLL_RESPONSE_WITH_MORE_1) client = create_client_11() gen = client.poll(POLL_COLLECTION, uri=POLL_PATH) block_1 = next(gen) assert block_1.content.decode('utf-8') == CONTENT_BLOCKS[0] message = get_sent_message() assert type(message) == tm11.PollRequest assert message.collection_name == POLL_COLLECTION responses.remove(responses.POST, POLL_URI) register_uri(POLL_URI, POLL_RESPONSE_WITH_MORE_2) block_2 = next(gen) assert block_2.content.decode('utf-8') == CONTENT_BLOCKS[1] message = get_sent_message() assert type(message) == tm11.PollFulfillmentRequest assert message.collection_name == POLL_COLLECTION assert message.result_part_number == 2 @responses.activate def test_poll_with_content_bindings(): register_uri(POLL_URI, POLL_RESPONSE_WITH_MORE_1) client = create_client_11() gen = client.poll(POLL_COLLECTION, uri=POLL_PATH, content_bindings=[CONTENT_BINDING]) block_1 = next(gen) assert block_1.content.decode('utf-8') == CONTENT_BLOCKS[0] message = get_sent_message() assert type(message) == tm11.PollRequest assert message.collection_name == POLL_COLLECTION poll_params = message.poll_parameters assert len(poll_params.content_bindings) == 1 assert poll_params.content_bindings[0].binding_id == CONTENT_BINDING subtypes = ['some-subtype-a', 'some-subtype-b'] binding = entities.ContentBinding(CONTENT_BINDING, subtypes=subtypes) gen = client.poll(POLL_COLLECTION, uri=POLL_PATH, content_bindings=[binding]) block_1 = next(gen) assert block_1.content.decode('utf-8') == CONTENT_BLOCKS[0] message = get_sent_message() assert type(message) == tm11.PollRequest poll_params = message.poll_parameters assert len(poll_params.content_bindings) == 1 assert poll_params.content_bindings[0].binding_id == binding.id assert poll_params.content_bindings[0].subtype_ids == binding.subtypes @responses.activate def test_subscribe(): register_uri(COLLECTION_MANAGEMENT_URI, SUBSCRIPTION_RESPONSE) client = create_client_11() response = client.subscribe(POLL_COLLECTION, uri=COLLECTION_MANAGEMENT_PATH) assert response.collection_name == POLL_COLLECTION message = get_sent_message() assert type(message) == tm11.ManageCollectionSubscriptionRequest assert message.collection_name == POLL_COLLECTION assert message.action == tm11.ACT_SUBSCRIBE @responses.activate def test_subscribe_with_push(): register_uri(COLLECTION_MANAGEMENT_URI, SUBSCRIPTION_RESPONSE) register_uri(DISCOVERY_URI_HTTP, DISCOVERY_RESPONSE) client = create_client_11(discovery_path=DISCOVERY_PATH) services = client.discover_services() inbox = next((s for s in services if s.type == SVC_INBOX), None) response = client.subscribe(POLL_COLLECTION, inbox_service=inbox, uri=COLLECTION_MANAGEMENT_PATH) assert response.collection_name == POLL_COLLECTION message = get_sent_message() assert type(message) == tm11.ManageCollectionSubscriptionRequest assert message.collection_name == POLL_COLLECTION assert message.push_parameters.inbox_address == inbox.address assert message.action == tm11.ACT_SUBSCRIBE @responses.activate def test_subscribtion_status(): register_uri(COLLECTION_MANAGEMENT_URI, SUBSCRIPTION_RESPONSE) client = create_client_11() response = client.get_subscription_status(POLL_COLLECTION, subscription_id=SUBSCRIPTION_ID, uri=COLLECTION_MANAGEMENT_PATH) assert response.collection_name == POLL_COLLECTION message = get_sent_message() assert type(message) == tm11.ManageCollectionSubscriptionRequest assert message.collection_name == POLL_COLLECTION assert message.action == tm11.ACT_STATUS @responses.activate def test_unsubscribe(): register_uri(COLLECTION_MANAGEMENT_URI, SUBSCRIPTION_RESPONSE) client = create_client_11() response = client.unsubscribe(POLL_COLLECTION, subscription_id=SUBSCRIPTION_ID, uri=COLLECTION_MANAGEMENT_PATH) assert response.collection_name == POLL_COLLECTION assert len(response.subscriptions) == 1 subscription = response.subscriptions[0] assert subscription.subscription_parameters.response_type == \ entities.SubscriptionParameters.TYPE_FULL message = get_sent_message() assert type(message) == tm11.ManageCollectionSubscriptionRequest assert message.collection_name == POLL_COLLECTION assert message.action == tm11.ACT_UNSUBSCRIBE @responses.activate def test_pause_subscription(): register_uri(COLLECTION_MANAGEMENT_URI, SUBSCRIPTION_RESPONSE) client = create_client_11() response = client.pause_subscription(POLL_COLLECTION, subscription_id=SUBSCRIPTION_ID, uri=COLLECTION_MANAGEMENT_PATH) assert response.collection_name == POLL_COLLECTION message = get_sent_message() assert type(message) == tm11.ManageCollectionSubscriptionRequest assert message.collection_name == POLL_COLLECTION assert message.action == tm11.ACT_PAUSE @responses.activate def test_resume_subscription(): register_uri(COLLECTION_MANAGEMENT_URI, SUBSCRIPTION_RESPONSE) client = create_client_11() response = client.resume_subscription(POLL_COLLECTION, subscription_id=SUBSCRIPTION_ID, uri=COLLECTION_MANAGEMENT_PATH) assert response.collection_name == POLL_COLLECTION message = get_sent_message() assert type(message) == tm11.ManageCollectionSubscriptionRequest assert message.collection_name == POLL_COLLECTION assert message.action == tm11.ACT_RESUME @responses.activate def test_push(): register_uri(INBOX_URI, INBOX_RESPONSE) client = create_client_11() client.push(CONTENT, CONTENT_BINDING, uri=INBOX_URI) message = get_sent_message() assert type(message) == tm11.InboxMessage assert len(message.content_blocks) == 1 assert message.content_blocks[0].content == CONTENT binding = message.content_blocks[0].content_binding.binding_id assert binding == CONTENT_BINDING @responses.activate def test_push_with_destination(): register_uri(INBOX_URI, INBOX_RESPONSE) client = create_client_11() dest_collections = [POLL_COLLECTION] client.push(CONTENT, CONTENT_BINDING, collection_names=dest_collections, uri=INBOX_URI) message = get_sent_message() assert type(message) == tm11.InboxMessage assert len(message.content_blocks) == 1 assert message.content_blocks[0].content == CONTENT binding = message.content_blocks[0].content_binding.binding_id assert binding == CONTENT_BINDING assert message.destination_collection_names == dest_collections cabby-0.1.23/tests/test_common.py000066400000000000000000000223451375517671000167420ustar00rootroot00000000000000import responses import pytest import json import gzip import sys import requests from time import sleep from six import StringIO from libtaxii import messages_11 as tm11 from libtaxii import messages_10 as tm10 from libtaxii.constants import ( VID_TAXII_XML_11, VID_TAXII_XML_10, ) from cabby import create_client from cabby import exceptions as exc import fixtures11 import fixtures10 CUSTOM_HEADER_NAME = 'X-custom-header' CUSTOM_HEADER_VALUE = 'header value with space!' JWT_PATH = '/management/auth/' JWT_URL = "http://example.localhost" + JWT_PATH def get_fix(version): return (fixtures10 if version == 10 else fixtures11) def make_client(version, **kwargs): client = create_client( get_fix(version).HOST, version=("1.1" if version == 11 else "1.0"), **kwargs) return client def register_uri(uri, body, version, headers=None): if headers is None: headers = {} headers.update(make_taxii_headers(version)) responses.add( method=responses.POST, url=uri, body=body, content_type="application/xml", stream=True, adding_headers=headers, ) def make_taxii_headers(version): return { "X-TAXII-Content-Type": VID_TAXII_XML_11 if version == 11 else VID_TAXII_XML_10 } def get_sent_message(version, mock=None): if not mock: mock = responses body = mock.calls[-1].request.body print(repr(body)) return (tm11 if version == 11 else tm10).get_message_from_xml(body) # Tests @pytest.mark.parametrize("version", [11, 10]) @responses.activate def test_set_headers(version): uri = get_fix(version).DISCOVERY_URI_HTTP response = get_fix(version).DISCOVERY_RESPONSE register_uri(uri, response, version) client = make_client( version, headers={CUSTOM_HEADER_NAME: CUSTOM_HEADER_VALUE}) services = client.discover_services(uri=uri) assert len(services) == 4 message = get_sent_message(version) expected_type = (tm11 if version == 11 else tm10).DiscoveryRequest assert type(message) == expected_type last_request = responses.calls[-1].request assert CUSTOM_HEADER_NAME in last_request.headers assert last_request.headers[CUSTOM_HEADER_NAME] == CUSTOM_HEADER_VALUE @pytest.mark.parametrize("version", [11, 10]) @responses.activate def test_invalid_response(version): uri = get_fix(version).DISCOVERY_URI_HTTP # FIXME: httpretty returns body as byte string (utf-8 encoded) # and when libtaxii tries to join headers (unicode) with the body (binary) # error happens. Line in Libtaxii codebase # https://github.com/EclecticIQ/libtaxii/blob/master/libtaxii/__init__.py#L126 return responses.add( method=responses.POST, url=uri, body='INVALID-BODY', content_type='text/html', ) client = make_client(version) with pytest.raises(exc.InvalidResponseError): client.discover_services(uri=uri) @pytest.mark.parametrize("version", [11, 10]) @responses.activate def test_invalid_response_status(version): uri = get_fix(version).DISCOVERY_URI_HTTP responses.add(method=responses.POST, url=uri, status=404) client = make_client(version) with pytest.raises(exc.HTTPError): client.discover_services(uri=uri) @pytest.mark.parametrize("version", [11, 10]) @responses.activate def test_jwt_auth_response(version): username = 'dummy-username' password = 'dummy-password' def jwt_request_callback(request): body = request.body if isinstance(body, bytes): body = body.decode() body = json.loads(body) assert body['username'] == username assert body['password'] == password content = json.dumps({"token": "dummy"}).encode() return (200, {}, content) # https://github.com/getsentry/responses/pull/268 responses.mock._matches.append(responses.CallbackResponse( method=responses.POST, url=JWT_URL, callback=jwt_request_callback, content_type='application/json', stream=True, )) discovery_uri = get_fix(version).DISCOVERY_URI_HTTP register_uri( discovery_uri, get_fix(version).DISCOVERY_RESPONSE, version) print(version, get_fix(version).DISCOVERY_RESPONSE) # client with relative JWT auth path client = make_client(version) client.set_auth( username=username, password=password, jwt_auth_url=JWT_PATH ) services = client.discover_services(uri=discovery_uri) assert len(services) == 4 # client with full JWT auth path client = make_client(version) client.set_auth( username=username, password=password, jwt_auth_url=JWT_URL ) services = client.discover_services(uri=discovery_uri) assert len(services) == 4 def compress(text): if sys.version_info < (3, 2): out = StringIO() with gzip.GzipFile(fileobj=out, mode="w") as f: f.write(text) return out.getvalue() else: return gzip.compress(text.encode('utf-8')) @pytest.mark.parametrize("version", [11, 10]) @responses.activate def test_gzip_response(version): uri = get_fix(version).DISCOVERY_URI_HTTP response = get_fix(version).DISCOVERY_RESPONSE body = compress(response) register_uri(uri, body, version, headers={ 'Content-Encoding': 'application/gzip' }) client = make_client(version) services = client.discover_services(uri=uri) assert len(services) == 4 @pytest.mark.parametrize("version", [11, 10]) @responses.activate def test_timeout(version): # this can't be tested with responses, # because it substitutes requests session # and doesn't worry about timeouts pytest.skip("cannot be tested yet") timeout_in_sec = 1 client = make_client(version) # configure to raise the error before the timeout client.timeout = timeout_in_sec / 2.0 def timeout_request_callback(request): sleep(timeout_in_sec) return (200, {'X-TAXII-Content-Type': content_type}, "All good!") uri = get_fix(version).DISCOVERY_URI_HTTP # https://github.com/getsentry/responses/pull/268 content_type = VID_TAXII_XML_11 if version == 11 else VID_TAXII_XML_10 responses.mock._matches.append(responses.CallbackResponse( responses.POST, uri, callback=timeout_request_callback, content_type='application/json', stream=True, )) with pytest.raises(requests.exceptions.Timeout): client.discover_services(uri=uri) @pytest.mark.parametrize("version", [11, 10]) @responses.activate def test_retry_once_on_unauthorized(version): # Test if the client refreshes the JWT if it receives an UNAUTHORIZED # status message. # Flow is as follows when client.poll() is called: # 1. Authenticate and get first_token # 2. Do poll request with first_token: Get UNAUTHORIZED response. # 3. Authenticate again and get second_token # 4. Do poll request with second_token: Get POLL_RESPONSE. # Set up two responses with tokens for auth request first_token = "first" second_token = "second" for token in (first_token, second_token): responses.add( method=responses.POST, url=JWT_URL, json={"token": token}, content_type="application/json", stream=True, ) client = make_client(version) client.set_auth(username="username", password="pass", jwt_auth_url=JWT_PATH) # Set up two responses for poll request: First is UNAUTHORIZED, the second is # a normal POLL_RESPONSE if the token was refreshed. attempts = [] def poll_callback(request): attempts.append(request) _, _, token = request.headers["Authorization"].partition("Bearer ") if len(attempts) == 1: assert token == first_token return ( 200, make_taxii_headers(version), get_fix(version).STATUS_MESSAGE_UNAUTHORIZED, ) else: assert len(attempts) == 2 assert token == second_token return (200, make_taxii_headers(version), get_fix(version).POLL_RESPONSE) responses.mock._matches.append( responses.CallbackResponse( responses.POST, url="http://example.localhost/poll", callback=poll_callback, stream=True, ) ) list(client.poll(collection_name="X", uri="/poll")) assert client.jwt_token == second_token @pytest.mark.parametrize("version", [11, 10]) @responses.activate def test_jwt_token_when_set_directly(version): given_token = "abcd" client = make_client(version) # The purpose of this test is to check that this assignment has effect: client.jwt_token = given_token def poll_callback(request): _, _, token = request.headers["Authorization"].partition("Bearer ") assert token == given_token return (200, make_taxii_headers(version), get_fix(version).POLL_RESPONSE) responses.mock._matches.append( responses.CallbackResponse( responses.POST, url="http://example.localhost/poll", callback=poll_callback, stream=True, ) ) list(client.poll(collection_name="X", uri="/poll")) cabby-0.1.23/tests/test_entities.py000066400000000000000000000037701375517671000172770ustar00rootroot00000000000000import pytest from cabby import entities @pytest.mark.parametrize('obj, expected', [ ( entities.ContentBlockCount(count=10, is_partial=False), 'ContentBlockCount(count=10, is_partial=False)', ), ( entities.Collection(name='test', description='not today'), 'Collection(name=test, type=DATA_FEED, available=None)', ), ( entities.ContentBinding(id=17), 'ContentBinding(id=17, subtypes=[])', ), ( entities.ServiceInstance( protocol='http', address='eiq', message_bindings=[], ), 'ServiceInstance(protocol=http, address=eiq)', ), ( entities.InboxService( protocol='http', address='eiq', message_bindings=[], ), 'InboxService(protocol=http, address=eiq)', ), ( entities.PushMethod(protocol='http', message_bindings=[]), 'PushMethod(protocol=http)', ), ( entities.SubscriptionParameters(response_type='FULL'), 'SubscriptionParameters(response_type=FULL)', ), ( entities.DetailedServiceInstance( type='test', version='urn:taxii.mitre.org:services:1.0', protocol='https', address='eiq', message_bindings=[]), 'DetailedServiceInstance(type=test, address=eiq)', ), ( entities.InboxDetailedService( type='test', version='urn:taxii.mitre.org:services:1.0', protocol='https', address='eiq', message_bindings=[], content_bindings=[]), 'InboxDetailedService(type=test, address=eiq)', ), ( entities.ContentBlock(content='', content_binding=[], timestamp=123), 'ContentBlock(timestamp=123)', ), ( entities.SubscriptionResponse(collection_name='eiq'), 'SubscriptionResponse(collection_name=eiq)', ), ( entities.Subscription(subscription_id=456), 'Subscription(subscription_id=456, status=UNKNOWN)', ), ]) def test_repr(obj, expected): assert repr(obj) == expected cabby-0.1.23/tests/test_ssl.py000066400000000000000000000067441375517671000162600ustar00rootroot00000000000000import socket import threading import pytest import werkzeug.serving import werkzeug.wrappers from six.moves import urllib import cabby import fixtures11 @pytest.fixture def httpsserver(request): port = get_free_port() server_key = 'tests/ssl_test_files/root_ca.key' server_cert = 'tests/ssl_test_files/root_ca.pem' server = werkzeug.serving.make_server( host='127.0.0.1', port=port, app=remote_taxii_app, passthrough_errors=True, ssl_context=(server_cert, server_key)) server_thread = threading.Thread(target=server.serve_forever) server_thread.start() request.addfinalizer(server.shutdown) return server def get_free_port(): s = socket.socket() s.bind(('', 0)) port = s.getsockname()[1] s.close() return port def remote_taxii_app(env, start_response): path = env['PATH_INFO'] if path == '/auth': start_response('200 OK', [('Content-Type', 'application/json')]) return [b'{"token": "123"}'] if path == fixtures11.DISCOVERY_PATH: taxii_headers = [ ('Content-Type', 'application/xml'), ('X-TAXII-Content-Type', 'urn:taxii.mitre.org:message:xml:1.1'), ] start_response('200 OK', taxii_headers) return [fixtures11.DISCOVERY_RESPONSE.encode()] raise Exception('Unknown test path') def test_set_auth_ssl(httpsserver): host, port = httpsserver.server_address client = cabby.create_client( host=host, port=port, use_https=True, discovery_path=fixtures11.DISCOVERY_PATH) auth_params = dict( username='cabby', password='test', ca_cert='tests/ssl_test_files/root_ca.pem', cert_file='tests/ssl_test_files/client.pem', key_file='tests/ssl_test_files/client.key', key_password='cabby-test', jwt_auth_url='/auth', verify_ssl=True) # Test client certificate with key_password. # This doesn't use the requests library, instead # it uses urllib2 or 3 through the 'request_with_key_password' function: client.set_auth(**auth_params) services = client.discover_services() assert len(services) == 4 # Test we get the correct error when forgetting a key_password: missing_key_password = dict(auth_params, key_password=None) client.set_auth(**missing_key_password) error_msg = "Key file is encrypted but key password was not provided" with pytest.raises(ValueError, match=error_msg): client.discover_services() # Test that server certificate authentication can fail: invalid_server_ca_cert = dict(auth_params, ca_cert=None) client.set_auth(**invalid_server_ca_cert) error_msg = "certificate verify failed" with pytest.raises(urllib.error.URLError, match=error_msg): client.discover_services() # Test that verify_ssl=False ignores the invalid certificate: invalid_server_ca_cert_no_verify = dict( auth_params, ca_cert=None, verify_ssl=False) client.set_auth(**invalid_server_ca_cert_no_verify) services = client.discover_services() assert len(services) == 4 # Test client certificate with key_password. # This uses the requests library as usual: client_key_without_passphrase = dict( auth_params, key_file='tests/ssl_test_files/client_no_pass.key', cert_file='tests/ssl_test_files/client_no_pass.pem') client.set_auth(**client_key_without_passphrase) services = client.discover_services() assert len(services) == 4 cabby-0.1.23/tox.ini000066400000000000000000000003631375517671000142060ustar00rootroot00000000000000[tox] envlist = flake8,py27,py36 [testenv] commands = py.test --cov {envsitepackagesdir}/cabby {posargs} deps = -rrequirements-dev.txt [testenv:flake8] basepython = python3 deps = flake8 commands = flake8 [flake8] exclude = docs/*,.tox