diff --git a/docs/conf.py b/docs/conf.py index f05a960..a757adf 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -23,6 +23,7 @@ extensions = [ "sphinx.ext.viewcode", "myst_parser", "sphinx_copybutton", + "sphinxcontrib.mermaid", ] diff --git a/docs/dynamic_records.md b/docs/dynamic_records.md deleted file mode 100644 index c0eeebf..0000000 --- a/docs/dynamic_records.md +++ /dev/null @@ -1,212 +0,0 @@ -# Dynamic Record Support - -Dynamic records provide support for GeoDNS and weighting to records. `A` and `AAAA` are fully supported and reasonably well tested for both Dyn (via Traffic Directors) and Route53. There is preliminary support for `CNAME` records, but caution should be exercised as they have not been thoroughly tested. - -Configuring GeoDNS is complex and the details of the functionality vary widely from provider to provider. octoDNS has an opinionated view mostly to give a reasonably consistent behavior across providers which is similar to the overall philosophy and approach of octoDNS itself. It may not fit your needs or use cases, in which case please open an issue for discussion. We expect this functionality to grow and evolve over time as it's more widely used. - -## An Annotated Example - -```yaml - ---- -test: - # This is a dynamic record when used with providers that support it - dynamic: - # These are the pools of records that can be referenced and thus used by rules - pools: - apac: - # An optional fallback, if all of the records in this pool fail this pool should be tried - fallback: na - # One or more values for this pool - values: - - value: 1.1.1.1 - - value: 2.2.2.2 - eu: - fallback: na - values: - - value: 3.3.3.3 - # Weight for this value, if omitted the default is 1 - weight: 2 - - value: 4.4.4.4 - weight: 3 - na: - # Implicitly goes to the backup pool (below) if all values are failing - # health checks - values: - - value: 5.5.5.5 - - value: 6.6.6.6 - - value: 7.7.7.7 - # Rules that assign queries to pools - rules: - - geos: - # Geos used in matching queries - - AF-ZA - - AS - - OC - # The pool to service the query from - pool: apac - - geos: - # AF-ZA was sent to apac above and the rest of AF else goes to eu here, - # sub-locations (e.g. AF-ZA) should come before their parents (AF.) If a - # more specific geo occured after a general one requests in that - # location would have already matched the previous rule. For the same - # reasons locations may not be repeated in multiple rules. - - AF - - EU - pool: eu - # No geos means match all queries, the final rule should generally be a - # "catch-all", served to any requests that didn't match the preceeding - # rules. The catch-all is the only case where a pool may be re-used. - - pool: na - ttl: 60 - type: A - # These values become a non-healthchecked backup/default pool, generally it - # should be a superset of the catch-all pool and include enough capacity to - # try and serve all global requests (with degraded performance.) The main - # case they will come into play is if all dynamic healthchecks are failing, - # either on the service side or if the providers systems are expeiencing - # problems. They will also be used for when the record is pushed to a - # provider that doesn't support dynamic records. - values: - - 3.3.3.3 - - 4.4.4.4 - - 5.5.5.5 - - 6.6.6.6 - - 7.7.7.7 -``` - -If you encounter validation errors in dynamic records suggesting best practices that you have specific reasons for not following see [records.md#Lenience](records.md#Lenience) for how to turn the errors into warnings. Doing so is taking on the burden of thoroughly testing and verifying that what you're doing behaves the way you expect. You may well encounter situations where the octoDNS providers and/or the underlying DNS services do not behave as desired. - -### Visual Representation of the Rules and Pools - -```mermaid ---- -title: Visual Representation of the Rules and Pools ---- -flowchart LR - query((Query)) --> rule_0[Rule 0
AF-ZA
AS
OC] - rule_0 --no match--> rule_1[Rule 1
AF
EU] - rule_1 --no match--> rule_2["Rule 2
(catch all)"] - - rule_0 --match--> pool_apac[Pool apac
1.1.1.1
2.2.2.2] - pool_apac --fallback--> pool_na - rule_1 --match--> pool_eu["Pool eu
3.3.3.3 (2/5)
4.4.4.4 (3/5)"] - pool_eu --fallback--> pool_na - rule_2 --> pool_na[Pool na
5.5.5.5
6.6.6.6
7.7.7.7] - pool_na --backup--> values[values
3.3.3.3
4.4.4.4
5.5.5.5
6.6.6.6
7.7.7.7] - - classDef queryColor fill:#3B67A8,color:#ffffff - classDef ruleColor fill:#D8F57A,color:#000000 - classDef poolColor fill:#F57261,color:#000000 - classDef valueColor fill:#498FF5,color:#000000 - - class query queryColor - class rule_0,rule_1,rule_2 ruleColor - class pool_apac,pool_eu,pool_na poolColor - class values valueColor -``` - - - -### Geo Codes - -Geo codes consist of one to three parts depending on the scope of the area being targeted. Examples of these look like: - -* 'NA-US-KY' - North America, United States, Kentucky -* 'NA-US' - North America, United States -* 'NA' - North America - -The first portion is the continent: - -* 'AF': 14, # Continental Africa -* 'AN': 17, # Continental Antarctica -* 'AS': 15, # Continental Asia -* 'EU': 13, # Continental Europe -* 'NA': 11, # Continental North America -* 'OC': 16, # Continental Australia/Oceania -* 'SA': 12, # Continental South America - -The second is the two-letter ISO Country Code https://en.wikipedia.org/wiki/ISO_3166-2 and the third is the ISO Country Code Subdivision as per https://en.wikipedia.org/wiki/ISO_3166-2:US. Change the code at the end for the country you are subdividing. Note that these may not always be supported depending on the providers in use. - -### Subnets - -Dynamic record rules also support subnet targeting in some providers: - -``` -... - rules: - - geos: - - AS - - OC - subnets: - # Subnets used in matching queries - - 5.149.176.0/24 - pool: apac -... -``` - -## Rule ordering - -octoDNS has validations in place to ensure that sources have the rules ordered from the most specific match to the least specific match per the following categories: - -1. Subnet-only rules -2. Subnet+Geo rules -3. Geo-only rules -4. Catch-all rule (with no subnet or geo matching) - -The first 3 categories are optional, while the last one is mandatory. - -Subnet targeting is considered more specific than geo targeting. This means that if there is a subnet rule match as well as a geo rule match, subnet match must take precedence. Provider implementations must ensure this behavior of targeting precedence. - -## Health Checks - -octoDNS will automatically configure the provider to monitor each IP and check for a 200 response for **https:///_dns**. - -These checks can be customized via the `healthcheck` configuration options. - -```yaml - ---- -test: - ... - octodns: - healthcheck: - host: my-host-name - path: /dns-health-check - port: 443 - protocol: HTTPS - ... -``` - -| Key | Description | Default | -|--|--|--| -| host | FQDN for host header and SNI | - | -| path | path to check | _dns | -| port | port to check | 443 | -| protocol | HTTP/HTTPS/TCP | HTTPS | - -Healthchecks can also be skipped for individual pool values. These values can be forced to always-serve or never-serve using the `status` flag. - -`status` flag is optional and accepts one of three possible values, `up`/`down`/`obey`, with `obey` being the default: - -```yaml -test: - ... - dynamic: - pools: - na: - values: - - value: 1.2.3.4 - status: down - - value: 2.3.4.5 - status: up - - value: 3.4.5.6 - # defaults to status: obey - ... -``` - -Support matrix: -* NS1 and Azure DNS support all 3 flag values -* All other dynamic-capable providers only support the default `obey` - -See "Health Check Options" in individual provider documentation for customization support. diff --git a/docs/geo_records.md b/docs/geo_records.md deleted file mode 100644 index 92fa559..0000000 --- a/docs/geo_records.md +++ /dev/null @@ -1,103 +0,0 @@ -# Geo Record Support - -Note: Geo DNS records are still supported for the time being, but it is still strongly encouraged that you look at [Dynamic Records](/docs/dynamic_records.md) instead as they are a superset of functionality. - -GeoDNS is currently supported for `A` and `AAAA` records on the Dyn (via Traffic Directors) and Route53 providers. Records with geo information pushed to providers without support for them will be managed as non-geo records using the base values. - -Configuring GeoDNS is complex and the details of the functionality vary widely from provider to provider. octoDNS has an opinionated view of how GeoDNS should be set up and does its best to map that to each provider's offering in a way that will result in similar behavior. It may not fit your needs or use cases, in which case please open an issue for discussion. We expect this functionality to grow and evolve over time as it's more widely used. - -The following is an example of GeoDNS with three entries NA-US-CA, NA-US-NY, OC-AU. octoDNS creates another one labeled 'default' with the details for the actual A record, This default record is the failover record if the monitoring check fails. - -```yaml ---- -? '' -: type: TXT - value: v=spf1 -all -test: - geo: - NA-US-NY: - - 111.111.111.1 - NA-US-CA: - - 111.111.111.2 - OC-AU: - - 111.111.111.3 - EU: - - 111.111.111.4 - ttl: 300 - type: A - value: 111.111.111.5 -``` - - -The geo labels breakdown based on: - -1. - - 'AF': 14, # Continental Africa - - 'AN': 17, # Continental Antarctica - - 'AS': 15, # Continental Asia - - 'EU': 13, # Continental Europe - - 'NA': 11, # Continental North America - - 'OC': 16, # Continental Australia/Oceania - - 'SA': 12, # Continental South America - -2. ISO Country Code https://en.wikipedia.org/wiki/ISO_3166-2 - -3. ISO Country Code Subdivision as per https://en.wikipedia.org/wiki/ISO_3166-2:US (change the code at the end for the country you are subdividing) * these may not always be supported depending on the provider. - -So the example is saying: - -- North America - United States - New York: gets served an "A" record of 111.111.111.1 -- North America - United States - California: gets served an "A" record of 111.111.111.2 -- Oceania - Australia: Gets served an "A" record of 111.111.111.3 -- Europe: gets an "A" record of 111.111.111.4 -- Everyone else gets an "A" record of 111.111.111.5 - -## Health Checks - -octoDNS will automatically set up monitors check for a 200 response for **https:///_dns**. - -These checks can be configured by adding a `healthcheck` configuration to the record: - -```yaml ---- -test: - geo: - AS: - - 1.2.3.4 - EU: - - 2.3.4.5 - octodns: - healthcheck: - host: my-host-name - path: /dns-health-check - port: 443 - protocol: HTTPS -``` - -| Key | Description | Default | -|--|--|--| -| host | FQDN for host header and SNI | - | -| path | path to check | _dns | -| port | port to check | 443 | -| protocol | HTTP/HTTPS | HTTPS | - -### Route53 Healtch Check Options - -| Key | Description | Default | -|--|--|--| -| measure_latency | Show latency in AWS console | true | -| request_interval | Healthcheck interval [10\|30] seconds | 10 | - -```yaml ---- - octodns: - healthcheck: - host: my-host-name - path: /dns-health-check - port: 443 - protocol: HTTPS - route53: - healthcheck: - measure_latency: false - request_interval: 30 -``` diff --git a/docs/source/api.rst b/docs/source/api.rst index 7d27920..088efa5 100644 --- a/docs/source/api.rst +++ b/docs/source/api.rst @@ -5,5 +5,7 @@ Developer Interface .. module:: octodns -.. autoclass:: octodns.provider.yaml.YamlProvider - :inherited-members: +.. toctree:: + :maxdepth: 1 + + /api/yaml-provider.rst diff --git a/docs/source/api/yaml-provider.rst b/docs/source/api/yaml-provider.rst new file mode 100644 index 0000000..86a81d8 --- /dev/null +++ b/docs/source/api/yaml-provider.rst @@ -0,0 +1,5 @@ +YamlProvider +============ + +.. autoclass:: octodns.provider.yaml.YamlProvider + :inherited-members: diff --git a/docs/assets/deploy.png b/docs/source/assets/deploy.png similarity index 100% rename from docs/assets/deploy.png rename to docs/source/assets/deploy.png diff --git a/docs/assets/noop.png b/docs/source/assets/noop.png similarity index 100% rename from docs/assets/noop.png rename to docs/source/assets/noop.png diff --git a/docs/assets/pr.png b/docs/source/assets/pr.png similarity index 100% rename from docs/assets/pr.png rename to docs/source/assets/pr.png diff --git a/docs/auto_arpa.md b/docs/source/auto_arpa.md similarity index 100% rename from docs/auto_arpa.md rename to docs/source/auto_arpa.md diff --git a/docs/source/configuration.rst b/docs/source/configuration.rst new file mode 100644 index 0000000..4a3e610 --- /dev/null +++ b/docs/source/configuration.rst @@ -0,0 +1,143 @@ +Configuration +============= + +Basics +------ + +This document picks up where :doc:`getting-started` and :doc:`records` leave off, +discussing details and less common scenarios. + +:doc:`api/yaml-provider` lays out the options for configuring the most commonly +used source of record data. + +Static Zone Config +.................. + +In cases where finer grained control is desired and the configuration of +individual zones varies ``zones`` can be an explicit list with each configured +zone listed along with its specific setup. As exemplified below ``alias`` zones +can be useful when two zones are exact copies of each other, with the same +configuration and records. YAML anchors are also helpful to avoid duplication +where zones share config, but not records.:: + + --- + manager: + include_meta: True + max_workers: 2 + + providers: + config: + class: octodns.provider.yaml.YamlProvider + directory: ./config + default_ttl: 3600 + enforce_order: True + ns1: + class: octodns_ns1.Ns1Provider + api_key: env/NS1_API_KEY + route53: + class: octodns_route53.Route53Provider + access_key_id: env/AWS_ACCESS_KEY_ID + secret_access_key: env/AWS_SECRET_ACCESS_KEY + + zones: + example.com.: &dual_target + sources: + - config + targets: + - ns1 + - route53 + + # these have the same setup as example.com., but will have their own files + # in the configuration directory for records. + third.tv.: *dual_target + fourth.tv.: *dual_target + + example.net.: + # example.net. is an exact copy of example.com., there will not be an + # example.net.yaml file in the config directory as `alias` includes + # duplicating the records of the aliased zone along with its config. + alias: example.com. + + other.com.: + lenient: True + sources: + - config + targets: + - ns1 + +General Configuration Concepts +.............................. + +``class`` is a special key that tells octoDNS what python class should be +loaded. Any other keys will be passed as configuration values to that +provider. In general any sensitive or frequently rotated values should come +from environmental variables. When octoDNS sees a value that starts with +``env/`` it will look for that value in the process's environment and pass the +result along. + +Further information can be found in the docstring of each source and provider +class. + +The ``include_meta`` key in the ``manager`` section of the config controls the +creation of a TXT record at the root of a zone that is managed by octoDNS. If +set to ``True``, octoDNS will create a TXT record for the root of the zone with +the value ``provider=``. If not specified, the default value for +``include_meta`` is ``False``. + +The ``max_workers`` key in the ``manager`` section of the config enables threading +to parallelize the planning portion of the sync. + +``lenient`` +........... + +``lenient`` mostly focuses on the details of ``Record``s and standards +compliance. When set to ``true`` octoDNS will allow non-compliant +configurations & values where possible. For example CNAME values that don't end +with a ``.``, label length restrictions, and invalid geo codes on ``dynamic`` +records. When in lenient mode octoDNS will log validation problems at +``WARNING`` and try and continue with the configuration or source data as it +exists. See Lenience_ for more information on the concept and how it can be +configured. + +.. _Lenience: records.rst#lenience + +``strict_supports`` +................... + +``strict_supports`` is a ``Provider`` level parameter that comes into play when +a provider has been asked to create a record that it is unable to support. The +simplest case of this would be record type, e.g. ``SSHFP`` not being supported +by ``AzureProvider``. If such a record is passed to an ``AzureProvider`` as a +target the provider will take action based on the ``strict_supports``. When +``true`` it will throw an exception saying that it's unable to create the +record, when set to ``false`` it will log at ``WARNING`` with information about +what it's unable to do and how it is attempting to work around it. Other +examples of things that cannot be supported would be ``dynamic`` records on a +provider that only supports simple or the lack of support for specific geos in +a provider, e.g. Route53Provider does not support ``NA-CA-*``. + +It is worth noting that these errors will happen during the plan phase of +things so that problems will be visible without having to make changes. + +As of octoDNS 1.x ``strict_supports`` is on by default. You have the choice to +set ``strict_supports=false`` on a per provider basis to request that things warn +and continue in a best-effort fashion. + +Configuring ``strict_supports`` +******************************* + +The ``strict_supports`` parameter is available on all providers and can be +configured in YAML as follows:: + + providers: + someprovider: + class: whatever.TheProvider + ... + strict_supports: true + +Automatic PTR generation +........................ + +octoDNS supports automatically generating PTR records from the ``A``/``AAAA`` +records it manages. For more information see the :doc:`auto_arpa` +documentation. diff --git a/docs/source/dynamic_records.rst b/docs/source/dynamic_records.rst new file mode 100644 index 0000000..ca85a2e --- /dev/null +++ b/docs/source/dynamic_records.rst @@ -0,0 +1,249 @@ +Dynamic Record Support +====================== + +Dynamic records provide support for GeoDNS and weighting to records. ``A`` and +``AAAA`` are fully supported and reasonably well tested for both Dyn (via +Traffic Directors) and Route53. There is preliminary support for ``CNAME`` +records, but caution should be exercised as they have not been thoroughly +tested. + +Configuring GeoDNS is complex and the details of the functionality vary widely +from provider to provider. octoDNS has an opinionated view mostly to give a +reasonably consistent behavior across providers which is similar to the overall +philosophy and approach of octoDNS itself. It may not fit your needs or use +cases, in which case please open an issue for discussion. We expect this +functionality to grow and evolve over time as it's more widely used. + +An Annotated Example +-------------------- + +:: + + --- + test: + # This is a dynamic record when used with providers that support it + dynamic: + # These are the pools of records that can be referenced and thus used by rules + pools: + apac: + # An optional fallback, if all of the records in this pool fail this pool should be tried + fallback: na + # One or more values for this pool + values: + - value: 1.1.1.1 + - value: 2.2.2.2 + eu: + fallback: na + values: + - value: 3.3.3.3 + # Weight for this value, if omitted the default is 1 + weight: 2 + - value: 4.4.4.4 + weight: 3 + na: + # Implicitly goes to the backup pool (below) if all values are failing + # health checks + values: + - value: 5.5.5.5 + - value: 6.6.6.6 + - value: 7.7.7.7 + # Rules that assign queries to pools + rules: + - geos: + # Geos used in matching queries + - AF-ZA + - AS + - OC + # The pool to service the query from + pool: apac + - geos: + # AF-ZA was sent to apac above and the rest of AF else goes to eu here, + # sub-locations (e.g. AF-ZA) should come before their parents (AF.) If a + # more specific geo occured after a general one requests in that + # location would have already matched the previous rule. For the same + # reasons locations may not be repeated in multiple rules. + - AF + - EU + pool: eu + # No geos means match all queries, the final rule should generally be a + # "catch-all", served to any requests that didn't match the preceeding + # rules. The catch-all is the only case where a pool may be re-used. + - pool: na + ttl: 60 + type: A + # These values become a non-healthchecked backup/default pool, generally it + # should be a superset of the catch-all pool and include enough capacity to + # try and serve all global requests (with degraded performance.) The main + # case they will come into play is if all dynamic healthchecks are failing, + # either on the service side or if the providers systems are expeiencing + # problems. They will also be used for when the record is pushed to a + # provider that doesn't support dynamic records. + values: + - 3.3.3.3 + - 4.4.4.4 + - 5.5.5.5 + - 6.6.6.6 + - 7.7.7.7 + +If you encounter validation errors in dynamic records suggesting best practices +that you have specific reasons for not following see :doc:records#lenience for +how to turn the errors into warnings. Doing so is taking on the burden of +thoroughly testing and verifying that what you're doing behaves the way you +expect. You may well encounter situations where the octoDNS providers and/or +the underlying DNS services do not behave as desired. + +Visual Representation of the Rules and Pools +............................................ + +.. mermaid:: + + flowchart LR + query((Query)) --> rule_0[Rule 0
AF-ZA
AS
OC] + rule_0 --no match--> rule_1[Rule 1
AF
EU] + rule_1 --no match--> rule_2["Rule 2
(catch all)"] + + rule_0 --match--> pool_apac[Pool apac
1.1.1.1
2.2.2.2] + pool_apac --fallback--> pool_na + rule_1 --match--> pool_eu["Pool eu
3.3.3.3 (2/5)
4.4.4.4 (3/5)"] + pool_eu --fallback--> pool_na + rule_2 --> pool_na[Pool na
5.5.5.5
6.6.6.6
7.7.7.7] + pool_na --backup--> values[values
3.3.3.3
4.4.4.4
5.5.5.5
6.6.6.6
7.7.7.7] + + classDef queryColor fill:#3B67A8,color:#ffffff + classDef ruleColor fill:#D8F57A,color:#000000 + classDef poolColor fill:#F57261,color:#000000 + classDef valueColor fill:#498FF5,color:#000000 + + class query queryColor + class rule_0,rule_1,rule_2 ruleColor + class pool_apac,pool_eu,pool_na poolColor + class values valueColor + +Geo Codes +......... + +Geo codes consist of one to three parts depending on the scope of the area +being targeted. Examples of these look like: + +* 'NA-US-KY' - North America, United States, Kentucky +* 'NA-US' - North America, United States +* 'NA' - North America + +The first portion is the continent: + +* 'AF': 14, # Continental Africa +* 'AN': 17, # Continental Antarctica +* 'AS': 15, # Continental Asia +* 'EU': 13, # Continental Europe +* 'NA': 11, # Continental North America +* 'OC': 16, # Continental Australia/Oceania +* 'SA': 12, # Continental South America + +The second is the two-letter ISO Country Code +https://en.wikipedia.org/wiki/ISO_3166-2 and the third is the ISO Country Code +Subdivision as per https://en.wikipedia.org/wiki/ISO_3166-2:US. Change the code +at the end for the country you are subdividing. Note that these may not always +be supported depending on the providers in use. + +Subnets +....... + +Dynamic record rules also support subnet targeting in some providers:: + + + ... + rules: + - geos: + - AS + - OC + subnets: + # Subnets used in matching queries + - 5.149.176.0/24 + pool: apac + ... + +Rule ordering +------------- + +octoDNS has validations in place to ensure that sources have the rules ordered +from the most specific match to the least specific match per the following +categories: + +1. Subnet-only rules +2. Subnet+Geo rules +3. Geo-only rules +4. Catch-all rule (with no subnet or geo matching) + +The first 3 categories are optional, while the last one is mandatory. + +Subnet targeting is considered more specific than geo targeting. This means +that if there is a subnet rule match as well as a geo rule match, subnet match +must take precedence. Provider implementations must ensure this behavior of +targeting precedence. + +Health Checks +------------- + +octoDNS will automatically configure the provider to monitor each IP and check +for a 200 response for **https:///_dns**. + +These checks can be customized via the ``healthcheck`` configuration options.:: + + --- + test: + ... + octodns: + healthcheck: + host: my-host-name + path: /dns-health-check + port: 443 + protocol: HTTPS + ... + + +.. list-table:: + :header-rows: 1 + + * - Key + - Description + - Default + * - host + - FQDN for host header and SNI + - + * - path + - path to check + - _dns + * - port + - port to check + - 443 + * - protocol + - HTTP/HTTPS/TCP + - HTTPS + +Healthchecks can also be skipped for individual pool values. These values can +be forced to always-serve or never-serve using the ``status`` flag. + +``status`` flag is optional and accepts one of three possible values, +``up``/``down``/``obey``, with ``obey`` being the default:: + + test: + ... + dynamic: + pools: + na: + values: + - value: 1.2.3.4 + status: down + - value: 2.3.4.5 + status: up + - value: 3.4.5.6 + # defaults to status: obey + ... + +Support matrix: + +* NS1 and Azure DNS support all 3 flag values +* All other dynamic-capable providers only support the default ``obey`` + +See "Health Check Options" in individual provider documentation for +customization support. diff --git a/docs/source/examples/README.md b/docs/source/examples/README.md deleted file mode 100644 index 134632d..0000000 --- a/docs/source/examples/README.md +++ /dev/null @@ -1,33 +0,0 @@ -# Examples - -* Getting started with a [basic octoDNS configuration](basic/) - new to octoDNS - this is the place to start. It'll walk you through the main pieces of DNS IaC - with octoDNS including the process of planning and applying changes. -* [Migrating to octoDNS](migrating-to-octodns/) - have an existing DNS setup - you'd like to bring into octoDNS check this example out right after - [basic](basic/). It'll walk you through the steps of using `octodns-dump` to - pull the existing data out of your provider into matching YAML config files on - disk. - -This is a WIP and will definitely be rough around the edges, but in the spirit -of not letting perfect get in the way of good enough, or really existing at -all. It's being uploaded as a starting point. PRs/suggestions welcome as are -ideas for other subjects to cover. - -## Running PowerDNS - -If you'd like to play around with running the examples in this directory -interactively you'll need a target for pushing data to. -[octodns-powerdns](https://github.com/octodns/octodns-powerdns) is the best -stand-alone option for this and the examples directory makes extensive use of -it. There is a [docker-compose.yml](docker-compose.yml) file that should get a -fully functional copy of PowerDNS backed my MySQL with the API enabled along -with other relivant functionality. For any of the examples that refer to the -local PowerDNS instance the following instructions below should get it up and -running. - -1. If you haven't already [install docker compose](https://docs.docker.com/compose/install/) -1. If you don't already have a copy of octoDNS checked out run `git clone https://github.com/octodns/octodns.git` -1. In a seperate terminal window or tab -1. cd into the examples directory `cd octodns/examples` -1. Run docker-compose up `docker-compose up`, this will start up MySQL and PowerDNS running them in the foreground with their logs printing to the terminal diff --git a/docs/source/examples/README.rst b/docs/source/examples/README.rst new file mode 100644 index 0000000..0cfa735 --- /dev/null +++ b/docs/source/examples/README.rst @@ -0,0 +1,45 @@ +Examples +======== + +* Getting started with a [basic octoDNS configuration](basic/README.md) - new + to octoDNS this is the place to start. It'll walk you through the main pieces + of DNS IaC with octoDNS including the process of planning and applying changes. +* [Migrating to octoDNS](migrating-to-octodns/README.md) - have an existing DNS + setup you'd like to bring into octoDNS check this example out right after + [basic](basic/README.md). It'll walk you through the steps of using + ``octodns-dump`` to pull the existing data out of your provider into matching + YAML config files on disk. + +This is a WIP and will definitely be rough around the edges, but in the spirit +of not letting perfect get in the way of good enough, or really existing at +all. It's being uploaded as a starting point. PRs/suggestions welcome as are +ideas for other subjects to cover. + +Running PowerDNS +---------------- + +If you'd like to play around with running the examples in this directory +interactively you'll need a target for pushing data to. `octodns-powerdns`_ is +the best stand-alone option for this and the examples directory makes extensive +use of it. There is a [docker-compose.yml](docker-compose.yml) file that should +get a fully functional copy of PowerDNS backed my MySQL with the API enabled +along with other relivant functionality. For any of the examples that refer to +the local PowerDNS instance the following instructions below should get it up +and running. + +.. _octodns-powerdns: https://github.com/octodns/octodns-powerdns + +1. If you haven't already `install docker compose`_ +1. If you don't already have a copy of octoDNS checked out run ``git clone https://github.com/octodns/octodns.git`` +1. In a seperate terminal window or tab +1. cd into the examples directory ``cd octodns/examples`` +1. Run docker-compose up ``docker-compose up``, this will start up MySQL and PowerDNS running them in the foreground with their logs printing to the terminal + +.. _install docker compose: https://docs.docker.com/compose/install/ + + +.. toctree:: + :maxdepth: 1 + + basic/README.md + migrating-to-octodns/README.md diff --git a/docs/source/examples/basic/README.md b/docs/source/examples/basic/README.md index c692a99..c8a2bf5 100644 --- a/docs/source/examples/basic/README.md +++ b/docs/source/examples/basic/README.md @@ -1,4 +1,4 @@ -# Basic octoDNS Setup +# Example Setup This is the starting point octoDNS config, it's pretty similar to what you might see for managing a set of personal domains or a small business. @@ -250,5 +250,5 @@ No changes were planned ## What's Next -* Check out [migrating to octoDNS](../migrating-to-octodns) for an example of how to create zone configuration YAML files from your existing provider's configuration -* For a complete list check out the [Examples Directory](../) +* Check out [migrating to octoDNS](../migrating-to-octodns/README.md) for an example of how to create zone configuration YAML files from your existing provider's configuration +* For a complete list check out the [Examples Directory](../README.rst) diff --git a/docs/source/examples/migrating-to-octodns/README.md b/docs/source/examples/migrating-to-octodns/README.md index 92b6d47..c542158 100644 --- a/docs/source/examples/migrating-to-octodns/README.md +++ b/docs/source/examples/migrating-to-octodns/README.md @@ -1,4 +1,4 @@ -# Migrating to octoDNS via octodns-dump +# Migrating to octoDNS Importing an existing DNS setup into octoDNS management is a very straightforward process and can generally be completed in minutes. @@ -55,7 +55,7 @@ just to have something to work with the the actual process that begins in the next step. It's not something you'd normally do when migrating to octoDNS. Step 0 is to get a local PowerDNS instance running. -Check out out [Running PowerDNS](../README.md#running-powerdns) for info on +Check out out [Running PowerDNS](../README.rst#running-powerdns) for info on starting that up. Once you've done that run the following. You can ignore the output and move on to the next step. @@ -66,10 +66,10 @@ output and move on to the next step. ## Running octodns-dump Once you have your configuration files and octoDNS installed you're ready to -dump your zone configs. Here we've assumed that the provider being used supports -`list_zones`, not all do. If you get an error to that effect see -[dynamic-zone-config](../dynamic-zone-config) for details on how to explicitly -list your zones in the config file. +dump your zone configs. Here we've assumed that the provider being used +supports `list_zones`, not all do. If you get an error to that effect see +[dynamic-zone-config](/getting-started#dynamic-zone-config) for +details on how to explicitly list your zones in the config file. We first tell octodns-dump where to find our config file. We then tell it that we want to use the output provider defined in that file named `config` rather @@ -136,7 +136,7 @@ This is especially true if you have any advanced records configured in your provider. octoDNS generally cannot convert records with functionality like weights and/or geo-coding enabled. It'll generally import a "simple" version of that record and indicate it's doing so with a WARNING log message. If you hit -this situation see [Dynamic Records](/docs/dynamic_records.md). +this situation see [Dynamic Records](/dynamic_records.rst). ## Running octodns-dump again, now with --lenient @@ -204,7 +204,7 @@ config. So for now we'll enable `lenient` on the SSHFP record by editing my-dumpable.com.yaml adding `octodns.lenient = true` as shown below. For more -details on see [lenience](/docs/records.md#lenience). +details on see [lenience](../../records.md#lenience). ```yaml ... @@ -259,7 +259,7 @@ If you see things in the plan output section it's time to triple check and make sure they're, hopefully minimal, modifications your OK with making. If the changes are due to advanced functionality you'll need to step back and -plan a careful migration over to [Dynamic Records](/docs/dynamic_records.md) +plan a careful migration over to [Dynamic Records](/dynamic_records.rst) which is beyond the scope of this example. ## What's Next @@ -267,5 +267,5 @@ which is beyond the scope of this example. So now you can commit your config and start managing you DNS with octoDNS rather than clicking buttons in UIs or using whatever you previous had used. -* Check out [octoDNS basic example](../basic) for an example of how to create zone configuration YAML files from your existing provider's configuration -* For a complete list check out the [Examples Directory](../) +* Check out [octoDNS basic example](../basic/README.md) for an example of how to create zone configuration YAML files from your existing provider's configuration +* For a complete list check out the [Examples Directory](../README.rst) diff --git a/docs/source/getting-started.rst b/docs/source/getting-started.rst index 1cf7970..e3c5dee 100644 --- a/docs/source/getting-started.rst +++ b/docs/source/getting-started.rst @@ -1,4 +1,4 @@ -Getting started +Getting Started =============== Workspace @@ -6,7 +6,8 @@ Workspace Running through the following commands will install the latest release of octoDNS and set up a place for your config files to live. To determine if -provider specific requirements are necessary see the providers_ below.:: +provider specific requirements are necessary see the :doc:`index#providers` +below.:: $ mkdir dns $ cd dns @@ -79,82 +80,6 @@ An example config would look something like:: - ns1 - route53 -Static Zone Config -.................. - -In cases where finer grained control is desired and the configuration of -individual zones varies ``zones`` can be an explicit list with each configured -zone listed along with its specific setup. As exemplified below ``alias`` zones -can be useful when two zones are exact copies of each other, with the same -configuration and records. YAML anchors are also helpful to avoid duplication -where zones share config, but not records.:: - - --- - manager: - include_meta: True - max_workers: 2 - - providers: - config: - class: octodns.provider.yaml.YamlProvider - directory: ./config - default_ttl: 3600 - enforce_order: True - ns1: - class: octodns_ns1.Ns1Provider - api_key: env/NS1_API_KEY - route53: - class: octodns_route53.Route53Provider - access_key_id: env/AWS_ACCESS_KEY_ID - secret_access_key: env/AWS_SECRET_ACCESS_KEY - - zones: - example.com.: &dual_target - sources: - - config - targets: - - ns1 - - route53 - - # these have the same setup as example.com., but will have their own files - # in the configuration directory for records. - third.tv.: *dual_target - fourth.tv.: *dual_target - - example.net.: - # example.net. is an exact copy of example.com., there will not be an - # example.net.yaml file in the config directory as `alias` includes - # duplicating the records of the aliased zone along with its config. - alias: example.com. - - other.com.: - lenient: True - sources: - - config - targets: - - ns1 - -General Configuration Concepts -.............................. - -``class`` is a special key that tells octoDNS what python class should be -loaded. Any other keys will be passed as configuration values to that -provider. In general any sensitive or frequently rotated values should come -from environmental variables. When octoDNS sees a value that starts with -``env/`` it will look for that value in the process's environment and pass the -result along. - -Further information can be found in the docstring of each source and provider -class. - -The ``include_meta`` key in the ``manager`` section of the config controls the -creation of a TXT record at the root of a zone that is managed by octoDNS. If -set to ``True``, octoDNS will create a TXT record for the root of the zone with -the value ``provider=``. If not specified, the default value for -``include_meta`` is ``False``. - -The ``max_workers`` key in the ``manager`` section of the config enables threading -to parallelize the planning portion of the sync. Quick Example Record .................... @@ -173,7 +98,7 @@ single ``A`` record at the top-level of the domain. - 1.2.3.4 - 1.2.3.5 -Further information can be found in Records_ Documentation +Further information can be found in :doc:`records` documentation. Noop ---- diff --git a/docs/source/index.rst b/docs/source/index.rst index 09ea49c..fb4c666 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -23,62 +23,12 @@ Documentation :maxdepth: 1 getting-started.rst - examples/basic/README.md - examples/migrating-to-octodns/README.md - -``lenient`` -........... - -``lenient`` mostly focuses on the details of ``Record``s and standards -compliance. When set to ``true`` octoDNS will allow non-compliant -configurations & values where possible. For example CNAME values that don't end -with a ``.``, label length restrictions, and invalid geo codes on ``dynamic`` -records. When in lenient mode octoDNS will log validation problems at -``WARNING`` and try and continue with the configuration or source data as it -exists. See Lenience_ for more information on the concept and how it can be -configured. - -.. _Lenience: records.rst#lenience - -``strict_supports`` -................... - -``strict_supports`` is a ``Provider`` level parameter that comes into play when -a provider has been asked to create a record that it is unable to support. The -simplest case of this would be record type, e.g. ``SSHFP`` not being supported -by ``AzureProvider``. If such a record is passed to an ``AzureProvider`` as a -target the provider will take action based on the ``strict_supports``. When -``true`` it will throw an exception saying that it's unable to create the -record, when set to ``false`` it will log at ``WARNING`` with information about -what it's unable to do and how it is attempting to work around it. Other -examples of things that cannot be supported would be ``dynamic`` records on a -provider that only supports simple or the lack of support for specific geos in -a provider, e.g. Route53Provider does not support ``NA-CA-*``. - -It is worth noting that these errors will happen during the plan phase of -things so that problems will be visible without having to make changes. - -As of octoDNS 1.x ``strict_supports`` is on by default. You have the choice to -set ``strict_supports=false`` on a per provider basis to request that things warn -and continue in a best-effort fashion. - -Configuring ``strict_supports`` -******************************* - -The ``strict_supports`` parameter is available on all providers and can be -configured in YAML as follows:: - - providers: - someprovider: - class: whatever.TheProvider - ... - strict_supports: true - -Automatic PTR generation -........................ - -octoDNS supports automatically generating PTR records from the ``A``/``AAAA`` -records it manages. For more information see the `auto-arpa`_ documentation. + records.md + configuration.rst + dynamic_records.rst + auto_arpa.rst + examples/README.rst + api.rst Providers --------- @@ -299,6 +249,7 @@ cannot be synced to. .. list-table:: :header-rows: 1 + :widths: 50 50 * - Source/Module - Notes @@ -339,7 +290,7 @@ Processors * - `AcmeManagingProcessor`_ - Useful when processes external to octoDNS are managing acme challenge DNS records, e.g. LetsEncrypt * - `AutoArpa`_ - - See `Automatic PTR generation`_ + - See :doc:`configuration#automatic-ptr-generation` * - `EnsureTrailingDots`_ - Processor that ensures ALIAS, CNAME, DNAME, MX, NS, PTR, and SRVs have trailing dots * - `ExcludeRootNsChanges`_ @@ -423,14 +374,6 @@ easily be included with no coordination beyond getting them into For examples of building third-party sources and providers, see `Related Projects and Resources`_ -API Documentation ------------------ - -.. toctree:: - :maxdepth: 2 - - api - Contributing ------------ diff --git a/docs/records.md b/docs/source/records.md similarity index 96% rename from docs/records.md rename to docs/source/records.md index d5f2cc4..73f380e 100644 --- a/docs/records.md +++ b/docs/source/records.md @@ -1,4 +1,4 @@ -# octoDNS records +# Records ## Record types @@ -29,8 +29,7 @@ Adding new record types to octoDNS is relatively straightforward, but will requi ## Advanced Record Support (GeoDNS, Weighting) -* [Dynamic Records](/docs/dynamic_records.md) - the preferred method for configuring geo-location, weights, and healthcheck based fallback between pools of services. -* [Geo Records](/docs/geo_records.md) - the original implementation of geo-location based records, now superseded by Dynamic Records (above) +* [Dynamic Records](/dynamic_records.rst) - the preferred method for configuring geo-location, weights, and healthcheck based fallback between pools of services. ## Config (`YamlProvider`) diff --git a/requirements-dev.txt b/requirements-dev.txt index 3b44544..46dab51 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -6,12 +6,12 @@ build==1.3.0 certifi==2025.8.3 cffi==1.17.1 changelet==0.1.0 -charset-normalizer==3.4.2 +charset-normalizer==3.4.3 click==8.1.8; python_version<'3.10' click==8.2.1; python_version>='3.10' cmarkgfm==2024.11.20 -coverage==7.10.2 -cryptography==45.0.5 +coverage==7.10.3 +cryptography==45.0.6 docutils==0.21.2 id==1.5.0 iniconfig==2.1.0 diff --git a/requirements-docs.txt b/requirements-docs.txt index 149e18f..91e6efe 100644 --- a/requirements-docs.txt +++ b/requirements-docs.txt @@ -11,7 +11,7 @@ babel==2.17.0 beautifulsoup4==4.13.4 furo==2025.7.19 imagesize==1.4.1 -mdit-py-plugins==0.4.2 +mdit-py-plugins==0.5.0 myst-parser==3.0.1; python_version<'3.10' myst-parser==4.0.1; python_version>='3.10' roman-numerals-py==3.1.0 @@ -23,6 +23,7 @@ sphinxcontrib-applehelp==2.0.0 sphinxcontrib-devhelp==2.0.0 sphinxcontrib-htmlhelp==2.1.0 sphinxcontrib-jsmath==1.0.1 +sphinxcontrib-mermaid==1.0.0 sphinxcontrib-qthelp==2.0.0 sphinxcontrib-serializinghtml==2.0.0 typing_extensions==4.14.1 diff --git a/setup.py b/setup.py index 5827ca7..a10a614 100644 --- a/setup.py +++ b/setup.py @@ -76,6 +76,7 @@ setup( 'furo>=2024.8.6', 'myst-parser>=4.0.1', 'sphinx-copybutton>=0.5.2', + 'sphinxcontrib-mermaid>=1.0.0', ), }, install_requires=(