Dyn will still match them, even while they're empty before we've had a
chance to add the respons pools to them which is BAD and caused a
medium-size outage (thankfully not as bad as it could have been.) Ideally
we'd use publish=False and stage things, but that seems broken in the client
lib, there's no way to send publish=N. I also tried sending the
response_pool_ids as part of the create calls and response pool config if
one didn't exist, but neither of those routes worked :-(