update docstrings

CONTRIBUTING.md

@@ -11,30 +11,30 @@ If you have questions, or you'd like to check with us before embarking on a majo
 This project uses the [GitHub Flow](https://guides.github.com/introduction/flow/). That means that the `main` branch is stable and new development is done in feature branches. Feature branches are merged into the `main` branch via a Pull Request.
 
 0. Fork and clone the repository
-0. Configure and install the dependencies: `./script/bootstrap`
-0. Activate virtual environment: `source env/bin/activate`
-0. Make sure the tests pass on your machine: `./script/test`
-0. Create a new branch: `git checkout -b my-branch-name`
-0. Make your change, add tests, and make sure the tests still pass
-0. Make sure that `./script/lint` passes without any warnings
-0. Run `./script/format` to make sure your changes follow Python's preferred
+1. Configure and install the dependencies: `./script/bootstrap`
+2. Activate virtual environment: `source env/bin/activate`
+3. Make sure the tests pass on your machine: `./script/test`
+4. Create a new branch: `git checkout -b my-branch-name`
+5. Make your change, add tests, and make sure the tests still pass
+6. Make sure that `./script/lint` passes without any warnings
+7. Run `./script/format` to make sure your changes follow Python's preferred
    coding style
-0. Run `./script/changelog create ...` to add a changelog entry to your PR
-0. Make sure that coverage is at :100:% `./script/coverage` and open `htmlcov/index.html`
-   * You can open a draft PR for :eyes: & discussion prior to this
-0. Push to your fork and submit a pull request
+8. Run `./script/changelog create ...` to add a changelog entry to your PR
+9. Make sure that coverage is at :100:% `./script/coverage` and open `htmlcov/index.html`
+   - You can open a draft PR for :eyes: & discussion prior to this
+10. Push to your fork and submit a pull request
 
 We will handle updating the version, tagging the release, and releasing the gem. Please don't bump the version or otherwise attempt to take on these administrative internal tasks as part of your pull request.
 
 Here are a few things you can do that will increase the likelihood of your pull request being accepted:
 
-* Follow [pep8](https://www.python.org/dev/peps/pep-0008/)
+- Follow [pep8](https://www.python.org/dev/peps/pep-0008/)
 
 - Write thorough tests. No PRs will be merged without :100:% code coverage. More than that tests should be very thorough and cover as many (edge) cases as possible. We're working with DNS here and bugs can have a major impact so we need to do as much as reasonably possible to ensure quality. While :100:% doesn't even begin to mean there are no bugs, getting there often requires close inspection & a relatively complete understanding of the code. More times than not the endeavor will uncover at least minor problems.
 
 - Bug fixes require specific tests covering the addressed behavior.
 
-- Write or update documentation. If you have added a feature or changed an existing one, please make appropriate changes to the docs. Doc-only PRs are always welcome.
+- Write or update documentation. If you have added a feature or changed an existing one, please make appropriate changes to the docs. Doc-only PRs are always welcome. Style guide: <https://numpydoc.readthedocs.io/en/latest/format.html#>
 
 - Keep your change as focused as possible. If there are multiple changes you would like to make that are not dependent upon each other, consider submitting them as separate pull requests.
 

octodns/processor/acme.py

@@ -11,7 +11,30 @@ class AcmeManagingProcessor(BaseProcessor):
     log = getLogger("AcmeManagingProcessor")
 
     def __init__(self, name):
-        """
+        """Manage or ignore ACME records.
+
+        **ACME records in the source:**
+            Will be marked with metadata so octodns knows they are managed by octodns.
+
+        **ACME records in the target:**
+            If the octodns mark **IS NOT** found, they will be ignored. So ACME records in the
+            target will not be updated or deleted.
+
+            If the octodns mark **IS** found, they will be treated like normal records.
+
+        Note
+        ----
+        This filter processes records with names starting with: `_acme-challenge`,
+        mostly used for the DNS-01 challenge.
+
+        For example: `_acme-challenge.foo.domain.com`
+
+        References
+        ---------
+        https://letsencrypt.org/docs/challenge-types/#dns-01-challenge
+
+        Example
+        -------
         .. code-block:: yaml
 
             processors:
@@ -20,8 +43,12 @@ class AcmeManagingProcessor(BaseProcessor):
 
             zones:
                 something.com.:
+                    sources:
+                        - x
                     processors:
-                        - acme2
+                        - acme
+                    targets:
+                        - y
         """
         super().__init__(name)
 

octodns/processor/filter.py

@@ -59,8 +59,8 @@ class _TypeBaseFilter(_FilterProcessor):
 class TypeAllowlistFilter(_TypeBaseFilter, AllowsMixin):
     """Only manage records of the specified type(s).
 
-    Example usage:
-
+    Example
+    -------
     .. code-block:: yaml
 
         processors:
@@ -91,8 +91,8 @@ class TypeAllowlistFilter(_TypeBaseFilter, AllowsMixin):
 class TypeRejectlistFilter(_TypeBaseFilter, RejectsMixin):
     """Ignore records of the specified type(s).
 
-    Example usage:
-
+    Example
+    -------
     .. code-block:: yaml
 
         processors:
@@ -150,8 +150,8 @@ class _NameBaseFilter(_FilterProcessor):
 class NameAllowlistFilter(_NameBaseFilter, AllowsMixin):
     """Only manage records with names that match the provider patterns
 
-    Example usage:
-
+    Example
+    -------
     .. code-block:: yaml
 
         processors:
@@ -188,8 +188,8 @@ class NameAllowlistFilter(_NameBaseFilter, AllowsMixin):
 class NameRejectlistFilter(_NameBaseFilter, RejectsMixin):
     """Reject managing records with names that match the provider patterns
 
-    Example usage:
-
+    Example
+    -------
     .. code-block:: yaml
 
         processors:
@@ -263,8 +263,8 @@ class _ValueBaseFilter(_FilterProcessor):
 class ValueAllowlistFilter(_ValueBaseFilter, AllowsMixin):
     """Only manage records with values that match the provider patterns
 
-    Example usage:
-
+    Example
+    -------
     .. code-block:: yaml
 
         processors:
@@ -302,8 +302,8 @@ class ValueAllowlistFilter(_ValueBaseFilter, AllowsMixin):
 class ValueRejectlistFilter(_ValueBaseFilter, RejectsMixin):
     """Reject managing records with names that match the provider patterns
 
-    Example usage:
-
+    Example
+    -------
     .. code-block:: yaml
 
         processors:
@@ -371,8 +371,8 @@ class NetworkValueAllowlistFilter(_NetworkValueBaseFilter, AllowsMixin):
     """Only manage A and AAAA records with values that match the provider patterns
     All other types will be left as-is.
 
-    Example usage:
-
+    Example
+    -------
     .. code-block:: yaml
 
         processors:
@@ -401,8 +401,8 @@ class NetworkValueRejectlistFilter(_NetworkValueBaseFilter, RejectsMixin):
     """Reject managing A and AAAA records with value matching a that match the provider patterns
     All other types will be left as-is.
 
-    Example usage:
-
+    Example
+    -------
     .. code-block:: yaml
 
         processors:
@@ -430,8 +430,8 @@ class NetworkValueRejectlistFilter(_NetworkValueBaseFilter, RejectsMixin):
 class IgnoreRootNsFilter(BaseProcessor):
     """Do not manage Root NS Records.
 
-    Example usage:
-
+    Example
+    -------
     .. code-block:: yaml
 
         processors:
@@ -462,8 +462,8 @@ class IgnoreRootNsFilter(BaseProcessor):
 class ExcludeRootNsChanges(BaseProcessor):
     """Do not allow root NS record changes
 
-    Example usage:
-
+    Example
+    -------
     .. code-block:: yaml
 
         processors:
@@ -512,8 +512,8 @@ class ExcludeRootNsChanges(BaseProcessor):
 class ZoneNameFilter(_FilterProcessor):
     """Filter or error on record names that contain the zone name
 
-    Example usage:
-
+    Example
+    -------
     .. code-block:: yaml
 
         processors:

octodns/processor/meta.py

@@ -39,6 +39,8 @@ class MetaProcessor(BaseProcessor):
     settings. Values are in the form `key=<value>`, e.g.
     `time=2023-09-10T05:49:04.246953`
 
+    Example
+    -------
     .. code-block:: yaml
 
         processors:

octodns/processor/restrict.py

@@ -18,8 +18,8 @@ class TtlRestrictionFilter(BaseProcessor):
     default maximum is 604800 (seven days.) allowed_ttls is only used when
     explicitly configured and min and max are ignored in that case.
 
-    Example usage:
-
+    Example
+    -------
     .. code-block:: yaml
 
         processors:

octodns/processor/spf.py

@@ -25,8 +25,8 @@ class SpfDnsLookupProcessor(BaseProcessor):
     """
     Validate that SPF values in TXT records are valid.
 
-    Example usage:
-
+    Example
+    -------
     .. code-block:: yaml
 
         processors:

octodns/provider/base.py

@@ -19,9 +19,9 @@ class BaseProvider(BaseSource):
     ):
         super().__init__(id)
         self.log.debug(
-            '__init__: id=%s, apply_disabled=%s, '
-            'update_pcent_threshold=%.2f, '
-            'delete_pcent_threshold=%.2f',
+            "__init__: id=%s, apply_disabled=%s, "
+            "update_pcent_threshold=%.2f, "
+            "delete_pcent_threshold=%.2f",
             id,
             apply_disabled,
             update_pcent_threshold,
@@ -33,7 +33,7 @@ class BaseProvider(BaseSource):
         self.strict_supports = strict_supports
 
     def _process_desired_zone(self, desired):
-        '''
+        """
         An opportunity for providers to modify the desired zone records before
         planning. `desired` is a "shallow" copy, see `Zone.copy` for more
         information
@@ -49,52 +49,52 @@ class BaseProvider(BaseSource):
         - Must call supports_warn_or_except with information about any changes
           that are made to have them logged or throw errors depending on the
           provider configuration.
-        '''
+        """
 
         for record in desired.records:
             if not self.supports(record):
-                msg = f'{record._type} records not supported for {record.fqdn}'
-                fallback = 'omitting record'
+                msg = f"{record._type} records not supported for {record.fqdn}"
+                fallback = "omitting record"
                 self.supports_warn_or_except(msg, fallback)
                 desired.remove_record(record)
-            elif getattr(record, 'dynamic', False):
+            elif getattr(record, "dynamic", False):
                 if self.SUPPORTS_DYNAMIC:
                     if not self.SUPPORTS_POOL_VALUE_STATUS:
                         # drop unsupported status flag
                         unsupported_pools = []
                         for _id, pool in record.dynamic.pools.items():
-                            for value in pool.data['values']:
-                                if value['status'] != 'obey':
+                            for value in pool.data["values"]:
+                                if value["status"] != "obey":
                                     unsupported_pools.append(_id)
                         if unsupported_pools:
-                            unsupported_pools = ','.join(unsupported_pools)
+                            unsupported_pools = ",".join(unsupported_pools)
                             msg = (
                                 f'"status" flag used in pools {unsupported_pools}'
-                                f' in {record.fqdn} is not supported'
+                                f" in {record.fqdn} is not supported"
                             )
                             fallback = (
-                                'will ignore it and respect the healthcheck'
+                                "will ignore it and respect the healthcheck"
                             )
                             self.supports_warn_or_except(msg, fallback)
                             record = record.copy()
                             for pool in record.dynamic.pools.values():
-                                for value in pool.data['values']:
-                                    value['status'] = 'obey'
+                                for value in pool.data["values"]:
+                                    value["status"] = "obey"
                             desired.add_record(record, replace=True)
 
                     if not self.SUPPORTS_DYNAMIC_SUBNETS:
                         subnet_rules = []
                         for i, rule in enumerate(record.dynamic.rules):
                             rule = rule.data
-                            if not rule.get('subnets'):
+                            if not rule.get("subnets"):
                                 continue
 
-                            msg = f'rule {i + 1} contains unsupported subnet matching in {record.fqdn}'
-                            if rule.get('geos'):
-                                fallback = 'using geos only'
+                            msg = f"rule {i + 1} contains unsupported subnet matching in {record.fqdn}"
+                            if rule.get("geos"):
+                                fallback = "using geos only"
                                 self.supports_warn_or_except(msg, fallback)
                             else:
-                                fallback = 'skipping the rule'
+                                fallback = "skipping the rule"
                                 self.supports_warn_or_except(msg, fallback)
 
                             subnet_rules.append(i)
@@ -106,8 +106,8 @@ class BaseProvider(BaseSource):
                             # drop subnet rules in reverse order so indices don't shift during rule deletion
                             for i in sorted(subnet_rules, reverse=True):
                                 rule = rules[i].data
-                                if rule.get('geos'):
-                                    del rule['subnets']
+                                if rule.get("geos"):
+                                    del rule["subnets"]
                                 else:
                                     del rules[i]
 
@@ -115,14 +115,14 @@ class BaseProvider(BaseSource):
                             pools = record.dynamic.pools
                             pools_seen = set()
                             for rule in record.dynamic.rules:
-                                pool = rule.data['pool']
+                                pool = rule.data["pool"]
                                 while pool:
                                     pools_seen.add(pool)
-                                    pool = pools[pool].data.get('fallback')
+                                    pool = pools[pool].data.get("fallback")
                             pools_unseen = set(pools.keys()) - pools_seen
                             for pool in pools_unseen:
                                 self.log.warning(
-                                    '%s: skipping pool %s which is rendered unused due to lack of support for subnet targeting',
+                                    "%s: skipping pool %s which is rendered unused due to lack of support for subnet targeting",
                                     record.fqdn,
                                     pool,
                                 )
@@ -130,20 +130,20 @@ class BaseProvider(BaseSource):
 
                             desired.add_record(record, replace=True)
                 else:
-                    msg = f'dynamic records not supported for {record.fqdn}'
-                    fallback = 'falling back to simple record'
+                    msg = f"dynamic records not supported for {record.fqdn}"
+                    fallback = "falling back to simple record"
                     self.supports_warn_or_except(msg, fallback)
                     record = record.copy()
                     record.dynamic = None
                     desired.add_record(record, replace=True)
             elif (
-                record._type == 'PTR'
+                record._type == "PTR"
                 and len(record.values) > 1
                 and not self.SUPPORTS_MULTIVALUE_PTR
             ):
                 # replace with a single-value copy
-                msg = f'multi-value PTR records not supported for {record.fqdn}'
-                fallback = f'falling back to single value, {record.value}'
+                msg = f"multi-value PTR records not supported for {record.fqdn}"
+                fallback = f"falling back to single value, {record.value}"
                 self.supports_warn_or_except(msg, fallback)
                 record = record.copy()
                 record.values = [record.value]
@@ -153,22 +153,21 @@ class BaseProvider(BaseSource):
         if self.SUPPORTS_ROOT_NS:
             if not record:
                 self.log.warning(
-                    'root NS record supported, but no record '
-                    'is configured for %s',
+                    "root NS record supported, but no record is configured for %s",
                     desired.decoded_name,
                 )
         else:
             if record:
                 # we can't manage root NS records, get rid of it
-                msg = f'root NS record not supported for {record.fqdn}'
-                fallback = 'ignoring it'
+                msg = f"root NS record not supported for {record.fqdn}"
+                fallback = "ignoring it"
                 self.supports_warn_or_except(msg, fallback)
                 desired.remove_record(record)
 
         return desired
 
     def _process_existing_zone(self, existing, desired):
-        '''
+        """
         An opportunity for providers to modify the existing zone records before
         planning. `existing` is a "shallow" copy, see `Zone.copy` for more
         information
@@ -185,52 +184,52 @@ class BaseProvider(BaseSource):
         - Must call supports_warn_or_except with information about any changes
           that are made to have them logged or throw errors depending on the
           provider configuration.
-        '''
+        """
 
         existing_root_ns = existing.root_ns
         if existing_root_ns and (
             not self.SUPPORTS_ROOT_NS or not desired.root_ns
         ):
             self.log.info(
-                'root NS record in existing, but not supported or '
-                'not configured; ignoring it'
+                "root NS record in existing, but not supported or "
+                "not configured; ignoring it"
             )
             existing.remove_record(existing_root_ns)
 
         return existing
 
     def _include_change(self, change):
-        '''
+        """
         An opportunity for providers to filter out false positives due to
         peculiarities in their implementation. E.g. minimum TTLs.
-        '''
+        """
         return True
 
     def _extra_changes(self, existing, desired, changes):
-        '''
+        """
         An opportunity for providers to add extra changes to the plan that are
         necessary to update ancillary record data or configure the zone. E.g.
         base NS records.
-        '''
+        """
         return []
 
     def _plan_meta(self, existing, desired, changes):
-        '''
+        """
         An opportunity for providers to indicate they have "meta" changes
         to the zone which are unrelated to records. Examples may include service
         plan changes, replication settings, and notes. The returned data is
         arbitrary/opaque to octoDNS, with the only requirement being that
         pprint.pformat can display it. A dict is recommended.
-        '''
+        """
         return None
 
     def supports_warn_or_except(self, msg, fallback):
         if self.strict_supports:
-            raise SupportsException(f'{self.id}: {msg}')
-        self.log.warning('%s; %s', msg, fallback)
+            raise SupportsException(f"{self.id}: {msg}")
+        self.log.warning("%s; %s", msg, fallback)
 
     def plan(self, desired, processors=[]):
-        self.log.info('plan: desired=%s', desired.decoded_name)
+        self.log.info("plan: desired=%s", desired.decoded_name)
 
         existing = Zone(desired.name, desired.sub_zones)
         exists = self.populate(existing, target=True, lenient=True)
@@ -238,7 +237,7 @@ class BaseProvider(BaseSource):
             # If your code gets this warning see Source.populate for more
             # information
             self.log.warning(
-                'Provider %s used in target mode did not return exists', self.id
+                "Provider %s used in target mode did not return exists", self.id
             )
 
         # Make a (shallow) copy of the desired state so that everything from
@@ -266,7 +265,7 @@ class BaseProvider(BaseSource):
         changes = [c for c in changes if self._include_change(c)]
         after = len(changes)
         if before != after:
-            self.log.info('plan:   filtered out %s changes', before - after)
+            self.log.info("plan:   filtered out %s changes", before - after)
 
         # allow the provider to add extra changes it needs
         extra = self._extra_changes(
@@ -274,8 +273,8 @@ class BaseProvider(BaseSource):
         )
         if extra:
             self.log.info(
-                'plan:   extra changes\n  %s',
-                '\n  '.join([str(c) for c in extra]),
+                "plan:   extra changes\n  %s",
+                "\n  ".join([str(c) for c in extra]),
             )
             changes += extra
 
@@ -293,25 +292,25 @@ class BaseProvider(BaseSource):
                 delete_pcent_threshold=self.delete_pcent_threshold,
                 meta=meta,
             )
-            self.log.info('plan:   %s', plan)
+            self.log.info("plan:   %s", plan)
             return plan
-        self.log.info('plan:   No changes')
+        self.log.info("plan:   No changes")
         return None
 
     def apply(self, plan):
-        '''
+        """
         Submits actual planned changes to the provider. Returns the number of
         changes made
-        '''
+        """
         if self.apply_disabled:
-            self.log.info('apply: disabled')
+            self.log.info("apply: disabled")
             return 0
 
         zone_name = plan.desired.decoded_name
         num_changes = len(plan.changes)
-        self.log.info('apply: making %d changes to %s', num_changes, zone_name)
+        self.log.info("apply: making %d changes to %s", num_changes, zone_name)
         self._apply(plan)
         return len(plan.changes)
 
     def _apply(self, plan):
-        raise NotImplementedError('Abstract base class, _apply method missing')
+        raise NotImplementedError("Abstract base class, _apply method missing")

octodns/provider/yaml.py

@@ -75,14 +75,14 @@ class YamlProvider(BaseProvider):
             # (optional, default False)
             disable_zonefile: false
 
-    .. warning::
 
-        When using this provider as a target any existing comments or formatting
-        in the zone files will be lost when changes are applyed.
+    Attention
+    ---------
+    When using this provider as a target any existing comments or formatting
+    in the zone files will be lost when changes are applyed.
 
     Split Details
     -------------
-
     All files are stored in a subdirectory matching the name of the zone
     (including the trailing .) of the directory config. It is a recommended
     best practice that the files be named RECORD.yaml, but all files are
@@ -101,7 +101,6 @@ class YamlProvider(BaseProvider):
 
     Overriding Values
     -----------------
-
     Overriding values can be accomplished using multiple yaml providers in the
     `sources` list where subsequent providers have `populate_should_replace`
     set to `true`. An example use of this would be a zone that you want to push
@@ -465,8 +464,8 @@ class YamlProvider(BaseProvider):
 
 class SplitYamlProvider(YamlProvider):
     """
-    .. deprecated::
-        DEPRECATED: Use YamlProvider with the split_extension parameter instead.
+    .. deprecated:: 2.0
+        Use YamlProvider with the split_extension parameter instead.
 
     When migrating the following configuration options would result in the same
     behavior as SplitYamlProvider
@@ -480,6 +479,8 @@ class SplitYamlProvider(YamlProvider):
          split_catchall: true
          disable_zonefile: true
 
+    Warning
+    -------
     TO BE REMOVED: 2.0
     """
 

octodns/source/tinydns.py

@@ -465,7 +465,10 @@ class TinyDnsFileSource(TinyDnsBaseSource):
             # (optional, default 3600)
             default_ttl: 3600
 
-    NOTE: timestamps & lo fields are ignored if present.
+    Note
+    ----
+    timestamps & lo fields are ignored if present.
+
 
     The source intends to conform to and fully support the official spec,
     https://cr.yp.to/djbdns/tinydns-data.html and the common patch/extensions to

script/build-docs

@@ -2,5 +2,6 @@
 
 set -xeuo pipefail
 
-rm -rf docs/_build
+#rm -rf docs/_build
+
 sphinx-build -M html docs docs/_build "${@}"