add markdown formatter to scripts/format so there is a clear markdown style in the whole project. also needed for further documentation as its written in markdown.
@ -10,18 +10,18 @@ If you have questions, or you'd like to check with us before embarking on a majo
This project uses the [GitHub Flow](https://guides.github.com/introduction/flow/). That means that the `main` branch is stable and new development is done in feature branches. Feature branches are merged into the `main` branch via a Pull Request.
0. Fork and clone the repository
1. Configure and install the dependencies: `./script/bootstrap`
03. Make sure the tests pass on your machine: `./script/test`
04. Create a new branch: `git checkout -b my-branch-name`
05. Make your change, add tests, and make sure the tests still pass
06. Make sure that `./script/lint` passes without any warnings
07. Run `./script/format` to make sure your changes follow Python's preferred
coding style
08. Run `./script/changelog create ...` to add a changelog entry to your PR
09. Make sure that coverage is at :100:% `./script/coverage` and open `htmlcov/index.html`
- You can open a draft PR for :eyes: & discussion prior to this
10. Push to your fork and submit a pull request
We will handle updating the version, tagging the release, and releasing the gem. Please don't bump the version or otherwise attempt to take on these administrative internal tasks as part of your pull request.
1. Include the extracted module in your python environment, e.g. if using Route53 that would require adding the `octodns_route53` module to your requirements.txt, setup.py, or similar.
1. Update the `class` value for your provider to the new path, e.g. again for Route53 that would be replacing `octodns.provider.route53.Route53Provider` with `octodns_route53.Route53Provider`
2. Update the `class` value for your provider to the new path, e.g. again for Route53 that would be replacing `octodns.provider.route53.Route53Provider` with `octodns_route53.Route53Provider`
The module required and provider class path for extracted providers can be found in the table above.
@ -320,7 +320,7 @@ The module required and provider class path for extracted providers can be found
Similar to providers, but can only serve to populate records into a zone, cannot be synced to.
@ -332,15 +332,15 @@ Similar to providers, but can only serve to populate records into a zone, cannot
### Notes
* ALIAS support varies a lot from provider to provider care should be taken to verify that your needs are met in detail.
* Dyn's UI doesn't allow editing or view of TTL, but the API accepts and stores the value provided, this value does not appear to be used when served
* Dnsimple's uses the configured TTL when serving things through the ALIAS, there's also a secondary TXT record created alongside the ALIAS that octoDNS ignores
* octoDNS itself supports non-ASCII character sets, but in testing Cloudflare is the only provider where that is currently functional end-to-end. Others have failures either in the client libraries or API calls
- ALIAS support varies a lot from provider to provider care should be taken to verify that your needs are met in detail.
- Dyn's UI doesn't allow editing or view of TTL, but the API accepts and stores the value provided, this value does not appear to be used when served
- Dnsimple's uses the configured TTL when serving things through the ALIAS, there's also a secondary TXT record created alongside the ALIAS that octoDNS ignores
- octoDNS itself supports non-ASCII character sets, but in testing Cloudflare is the only provider where that is currently functional end-to-end. Others have failures either in the client libraries or API calls
| [AcmeManagingProcessor](/octodns/processor/acme.py) | Useful when processes external to octoDNS are managing acme challenge DNS records, e.g. LetsEncrypt |
| [AutoArpa](/octodns/processor/arpa.py) | See [Automatic PTR generation](#automatic-ptr-generation) below |
| [EnsureTrailingDots](/octodns/processor/trailing_dots.py) | Processor that ensures ALIAS, CNAME, DNAME, MX, NS, PTR, and SRVs have trailing dots |
@ -390,7 +390,7 @@ providers:
## Custom Sources and Providers
You can check out the [source](/octodns/source/) and [provider](/octodns/provider/) directory to see what's currently supported. Sources act as a source of record information. AxfrSource and TinyDnsFileSource are currently the only OSS sources, though we have several others internally that are specific to our environment. These include something to pull host data from [gPanel](https://githubengineering.com/githubs-metal-cloud/) and a similar provider that sources information about our network gear to create both `A`&`PTR` records for their interfaces. Things that might make good OSS sources might include an `ElbSource` that pulls information about [AWS Elastic Load Balancers](https://aws.amazon.com/elasticloadbalancing/) and dynamically creates `CNAME`s for them, or `Ec2Source` that pulls instance information so that records can be created for hosts similar to how our `GPanelProvider` works.
You can check out the [source](/octodns/source/) and [provider](/octodns/provider/) directory to see what's currently supported. Sources act as a source of record information. AxfrSource and TinyDnsFileSource are currently the only OSS sources, though we have several others internally that are specific to our environment. These include something to pull host data from [gPanel](https://githubengineering.com/githubs-metal-cloud/) and a similar provider that sources information about our network gear to create both `A`&`PTR` records for their interfaces. Things that might make good OSS sources might include an `ElbSource` that pulls information about [AWS Elastic Load Balancers](https://aws.amazon.com/elasticloadbalancing/) and dynamically creates `CNAME`s for them, or `Ec2Source` that pulls instance information so that records can be created for hosts similar to how our `GPanelProvider` works.
Most of the things included in octoDNS are providers, the obvious difference being that they can serve as both sources and targets of data. We'd really like to see this list grow over time so if you use an unsupported provider then PRs are welcome. The existing providers should serve as reasonable examples. Those that have no GeoDNS support are relatively straightforward. Unfortunately most of the APIs involved to do GeoDNS style traffic management are complex and somewhat inconsistent so adding support for that function would be nice, but is optional and best done in a separate pass.
@ -460,7 +460,9 @@ If you have a problem or suggestion, please [open an issue](https://github.com/o
That would take all the relevant records from example.com and add them as PTR records for the arpa zones in the same place as the 'config' source specifies.
Geo codes consist of one to three parts depending on the scope of the area being targeted. Examples of these look like:
* 'NA-US-KY' - North America, United States, Kentucky
* 'NA-US' - North America, United States
* 'NA' - North America
- 'NA-US-KY' - North America, United States, Kentucky
- 'NA-US' - North America, United States
- 'NA' - North America
The first portion is the continent:
* 'AF': 14, # Continental Africa
* 'AN': 17, # Continental Antarctica
* 'AS': 15, # Continental Asia
* 'EU': 13, # Continental Europe
* 'NA': 11, # Continental North America
* 'OC': 16, # Continental Australia/Oceania
* 'SA': 12, # Continental South America
- 'AF': 14, # Continental Africa
- 'AN': 17, # Continental Antarctica
- 'AS': 15, # Continental Asia
- 'EU': 13, # Continental Europe
- 'NA': 11, # Continental North America
- 'OC': 16, # Continental Australia/Oceania
- 'SA': 12, # Continental South America
The second is the two-letter ISO Country Code https://en.wikipedia.org/wiki/ISO_3166-2 and the third is the ISO Country Code Subdivision as per https://en.wikipedia.org/wiki/ISO_3166-2:US. Change the code at the end for the country you are subdividing. Note that these may not always be supported depending on the providers in use.
@ -160,7 +158,7 @@ Subnet targeting is considered more specific than geo targeting. This means that
### Health Checks
octoDNS will automatically configure the provider to monitor each IP and check for a 200 response for **https://<ip_address>/_dns**.
octoDNS will automatically configure the provider to monitor each IP and check for a 200 response for **https://\<ip_address>/\_dns**.
These checks can be customized via the `healthcheck` configuration options.
@ -178,10 +176,10 @@ test:
...
```
| Key | Description | Default |
|--|--|--|
| Key | Description | Default |
|--|--|--|
| host | FQDN for host header and SNI | - |
| path | path to check | _dns |
| path | path to check | \_dns |
| port | port to check | 443 |
| protocol | HTTP/HTTPS/TCP | HTTPS |
@ -206,7 +204,8 @@ test:
```
Support matrix:
* NS1 and Azure DNS support all 3 flag values
* All other dynamic-capable providers only support the default `obey`
- NS1 and Azure DNS support all 3 flag values
- All other dynamic-capable providers only support the default `obey`
See "Health Check Options" in individual provider documentation for customization support.
2. ISO Country Code https://en.wikipedia.org/wiki/ISO_3166-2
3. ISO Country Code Subdivision as per https://en.wikipedia.org/wiki/ISO_3166-2:US (change the code at the end for the country you are subdividing) * these may not always be supported depending on the provider.
3. ISO Country Code Subdivision as per https://en.wikipedia.org/wiki/ISO_3166-2:US (change the code at the end for the country you are subdividing) * these may not always be supported depending on the provider.
So the example is saying:
- North America - United States - New York: gets served an "A" record of 111.111.111.1
- North America - United States - California: gets served an "A" record of 111.111.111.2
- North America - United States - New York: gets served an "A" record of 111.111.111.1
- North America - United States - California: gets served an "A" record of 111.111.111.2
- Oceania - Australia: Gets served an "A" record of 111.111.111.3
- Europe: gets an "A" record of 111.111.111.4
- Everyone else gets an "A" record of 111.111.111.5
### Health Checks
octoDNS will automatically set up monitors check for a 200 response for **https://<ip_address>/_dns**.
octoDNS will automatically set up monitors check for a 200 response for **https://\<ip_address>/\_dns**.
These checks can be configured by adding a `healthcheck` configuration to the record:
@ -74,17 +72,17 @@ test:
protocol: HTTPS
```
| Key | Description | Default |
|--|--|--|
| Key | Description | Default |
|--|--|--|
| host | FQDN for host header and SNI | - |
| path | path to check | _dns |
| path | path to check | \_dns |
| port | port to check | 443 |
| protocol | HTTP/HTTPS | HTTPS |
#### Route53 Healtch Check Options
| Key | Description | Default |
|--|--|--|
| Key | Description | Default |
|--|--|--|
| measure_latency | Show latency in AWS console | true |
Underlying provider support for each of these varies and some providers have extra requirements or limitations. In cases where a record type is not supported by a provider octoDNS will ignore it there and continue to manage the record elsewhere. For example `SSHFP` is supported by Dyn, but not Route53. If your source data includes an SSHFP record octoDNS will keep it in sync on Dyn, but not consider it when evaluating the state of Route53. The best way to find out what types are supported by a provider is to look for its `supports` method. If that method exists the logic will drive which records are supported and which are ignored. If the provider does not implement the method it will fall back to `BaseProvider.supports` which indicates full support.
@ -29,8 +29,8 @@ Adding new record types to octoDNS is relatively straightforward, but will requi
## Advanced Record Support (GeoDNS, Weighting)
* [Dynamic Records](/docs/dynamic_records.md) - the preferred method for configuring geo-location, weights, and healthcheck based fallback between pools of services.
* [Geo Records](/docs/geo_records.md) - the original implementation of geo-location based records, now superseded by Dynamic Records (above)
- [Dynamic Records](/docs/dynamic_records.md) - the preferred method for configuring geo-location, weights, and healthcheck based fallback between pools of services.
- [Geo Records](/docs/geo_records.md) - the original implementation of geo-location based records, now superseded by Dynamic Records (above)
## Config (`YamlProvider`)
@ -96,6 +96,7 @@ octoDNS is fairly strict in terms of standards compliance and is opinionated in
It's best to think of the `lenient` flag as "I know what I'm doing and accept any problems I run across." The main reason being is that some providers may allow the non-compliant setup and others may not. The behavior of the non-compliant records may even vary from one provider to another. Caveat emptor.
#### Record priority for AutoArpa
When multiple A or AAAA records point to the same IP, it is possible to set an optional priority on each record. The records with the lowest priority will have the highest preference when being processed by AutoArpa. The AutoArpa provider will create PTR records in order of preference, up to a set limit defined by the `max_auto_arpa` option in the provider configuration.
```yaml
@ -143,12 +144,12 @@ If left unconfigured, suitable defaults take over instead. In the below example,
the Dyn provider is configured with limits of 40% on both update and
delete operations over all the records present.
````yaml
```yaml
dyn:
class: octodns.provider.dyn.DynProvider
update_pcent_threshold: 0.4
delete_pcent_threshold: 0.4
````
```
Additionally, thresholds can be configured at the zone level. Zone thresholds
take precedence over any provider default or explicit configuration. Zone
@ -7,10 +7,10 @@ ideas for other subjects to cover.
### Examples
* Getting started with a [basic octoDNS configuration](basic/) - new to octoDNS
- Getting started with a [basic octoDNS configuration](basic/) - new to octoDNS
this is the place to start. It'll walk you through the main pieces of DNS IaC
with octoDNS including the process of planning and applying changes.
* [Migrating to octoDNS](migrating-to-octodns/) - have an existing DNS setup
- [Migrating to octoDNS](migrating-to-octodns/) - have an existing DNS setup
you'd like to bring into octoDNS check this example out right after
[basic](basic/). It'll walk you through the steps of using `octodns-dump` to
pull the existing data out of your provider into matching YAML config files on
@ -29,7 +29,7 @@ local PowerDNS instance the following instructions below should get it up and
running.
1. If you haven't already [install docker compose](https://docs.docker.com/compose/install/)
1. If you don't already have a copy of octoDNS checked out run `git clone https://github.com/octodns/octodns.git`
1. In a seperate terminal window or tab
1. cd into the examples directory `cd octodns/examples`
1. Run docker-compose up `docker-compose up`, this will start up MySQL and PowerDNS running them in the foreground with their logs printing to the terminal
2. If you don't already have a copy of octoDNS checked out run `git clone https://github.com/octodns/octodns.git`
3. In a seperate terminal window or tab
4. cd into the examples directory `cd octodns/examples`
5. Run docker-compose up `docker-compose up`, this will start up MySQL and PowerDNS running them in the foreground with their logs printing to the terminal
From here on this README focuses on the general process of running octoDNS.
@ -250,5 +250,5 @@ No changes were planned
## What's Next
* Check out [migrating to octoDNS](../migrating-to-octodns) for an example of how to create zone configuration YAML files from your existing provider's configuration
* For a complete list check out the [Examples Directory](../)
- Check out [migrating to octoDNS](../migrating-to-octodns) for an example of how to create zone configuration YAML files from your existing provider's configuration
- For a complete list check out the [Examples Directory](../)
From here on this README focuses on the process of using `octodns-dump` to
import your existing DNS data into octoDNS.
@ -267,5 +267,5 @@ which is beyond the scope of this example.
So now you can commit your config and start managing you DNS with octoDNS rather
than clicking buttons in UIs or using whatever you previous had used.
* Check out [octoDNS basic example](../basic) for an example of how to create zone configuration YAML files from your existing provider's configuration
* For a complete list check out the [Examples Directory](../)
- Check out [octoDNS basic example](../basic) for an example of how to create zone configuration YAML files from your existing provider's configuration
- For a complete list check out the [Examples Directory](../)
Ivan Schaller
6 months ago