diff --git a/metadata/README.md b/metadata/README.md index 593200756..40a8990a4 100644 --- a/metadata/README.md +++ b/metadata/README.md @@ -11,6 +11,11 @@ generating the XML files will be released here, at which time the CSV files will become the source of truth for libphonenumber and the XML-based metadata and other mapping files will be automatically derived from it. +## About metadata.zip + +It contains canonical metadata for the libphonenumber project, intended for use +by libphonenumber tools. CSV schemas are not promised to be stable. + ### Support and Api Stability **These libraries are not currently supported and do not provide a stable API; diff --git a/metadata/metadata.zip b/metadata/metadata.zip new file mode 100644 index 000000000..549229b8e Binary files /dev/null and b/metadata/metadata.zip differ diff --git a/migrator/README.md b/migrator/README.md new file mode 100644 index 000000000..e36b78f29 --- /dev/null +++ b/migrator/README.md @@ -0,0 +1,271 @@ +# Migrator Library + +The library takes in E.164 phone numbers with their corresponding [BCP-47 country code](https://countrycode.org/). If inputted numbers match any of the supported internal renumbering ‘recipes’, which are created from official country documentation, said numbers will undergo a process of migration to convert them into the most up to date, dialable format. A given “recipe” is an internal record which holds data needed to migrate a stale number into one that is valid and dialable. + +## Capabilities + +The migrator library can be used as either a [command line tool](#command-line-usage) or a [web application](#web-application). Both applications have the following capabilities: + +```Single Number Migration``` - input a single E.164 phone number with its corresponding BCP-47 country code. If there is an available migration recipe, the number will be converted to the new format based on the given recipe. The resulting string of the migration will be outputted to the console. + +```File Migration``` - input a text file location with one E.164 number per line along with the BCP-47 country code that corresponds to the numbers in the text file. All numbers in the text file that match available recipes will be migrated and a copy of the text file will be created with the updated numbers. The file path to the newly created text file will be outputted to the console. + +```Custom Recipe Migration``` - specify a [custom recipe](#custom-recipe-migrations) file to use for a migration instead of the internal maintained file. Custom recipe migrations will not check if converted numbers are valid based on country formats. + +```Migration Report``` - find all phone numbers in a given input that were migrated with given migration recipes. The report will also show the phone numbers that were already seen as valid before migrations as well as identify any numbers that were migrated into invalid formats based on known valid phone number ranges for the specified country code. + +## Command Line Usage + +### Required Parameters + +```(-n, --number | -f, --file)``` - either the phone number input or file input for a given migration. The number argument must be a single E.164 phone number. This number can be in strict E.164 format (“+841200000000”) or contain hyphens, spaces and curved brackets (“84 (120) 5555-555”). The file argument must be a path to a text file containing one E.164 phone number per line. For file migrations, the original text file will not be overwritten or stored and instead a new text file will be created containing the migrated version of each phone number or the original phone number when a migration did not take place. Original phone numbers will also be written to file when a given migration on a number is seen as invalid, producing a number not dialable for the given country, unless the --exportInvalidMigrations (see [Optional Parameters](#optional-parameters)) flag is specified. Note: all phone numbers entered for migration will be sanitized by removing any whitespace, hyphens or curved brackets. If the number after this process is still not in E.164 format, a migration will not be able to be performed on it. + +```-c, --countryCode``` - the BCP-47 country code that corresponds to the given phone number input. Only recipes from this country will be queried to find matching recipes for inputted numbers. (E.g. 44 for United Kingdom, 1 for US, 84 for Vietnam) + +### Optional Parameters + +```-r, --customRecipe``` - the path to a csv file containing custom migration recipes which follow the standard recipes file format. When using custom recipes, validity checks on migrated numbers will not be performed. + +```-e, --exportInvalidMigrations``` - the boolean command line flag specifying that text files created after the migration process for standard file migration job should contain the migrated version of a phone number when a migration has taken place on it, regardless of whether the migration resulted in an invalid phone number. By default, when a migrated number is seen as invalid, its original inputted representation is written to file. Note: this flag has no effect on a --number migration or a migration using a --customRecipe. + +```-h, --help``` - usage help + + +### Installation and Setup + +Start off by using git to download the latest version of the libphonenumber repository: + +```bash +git clone https://github.com/google/libphonenumber.git +cd libphonenumber +ls +``` + +The project must then be built with its dependencies using maven so that an executable jar can be created and run: + +```bash +# clean install metadata dependency to run migrator tool +cd metadata +mvn clean install + +cd ../migrator +# clean install migrator to run migrator-servlet locally (optional) +mvn clean install + +mvn clean compile assembly:single +cd target +ls +``` + +### Single Number Migration + +To perform a single number migration, the “countryCode” and “number” arguments must be specified. The tool will clean the number string to remove spaces, hyphens and curved brackets. Note that when a number contains spaces, the whole string must be enclosed with quotation marks. + +An example of a valid migration of a Vietnam phone number is as shown below: + +```bash +java -jar migrator-1.0.0-jar-with-dependencies.jar --countryCode=84 --number=+841201234567 + +# Output: +# Migration of country code +84 phone number(s): +# Successful migration into: +84701234567 +``` + +If the given number is already in a valid format for the given country code, a migration will not occur and instead a message explaining this will be printed: + +```bash +java -jar migrator-1.0.0-jar-with-dependencies.jar --countryCode=84 --number=+84(70)123-4567 + +# Output: +# Migration of country code +84 phone number(s): +# This number was seen to already be valid and dialable based on our data for the given country +``` + +The migrator tool will not migrate or validate a number that belongs to a different country from the specified country code. In such situations the tool will take no action: + +```bash +java -jar migrator-1.0.0-jar-with-dependencies.jar --countryCode=84 --number=”+44 77 12345 678” + +# Output: +# Migration of country code +84 phone number(s): +# This number could not be migrated using any of the recipes from the given recipes file +``` + +### File Migration + +To perform a file migration, the “countryCode” and “file” arguments must be specified. The tool will generate a new text file with the outputted numbers once the migration is completed. For the following examples, “VietnamNumbers.txt” is a text file containing the phone numbers: + +```bash +VietnamNumbers.txt + +84-1201234567 ++84(121)7654321 ++841225555555 +84 120 1424534 ++84709000000 ++123829734972 +373203934781 +``` +##### Migration using the country code +84: +```bash +java -jar migrator-1.0.0-jar-with-dependencies.jar --countryCode=84 --file=../VietnamNumbers.txt + +# Output: +# Migration of country code +84 phone number(s): +# New numbers file created at: ./+84_Migration_VietnamNumbers.txt +``` +```bash ++84_Migration_VietnamNumbers.txt + ++84701234567 ++84797654321 ++84775555555 ++84701424534 ++84709000000 ++123829734972 +373203934781 +``` +##### Migration using the country code +1: +```bash +java -jar migrator-1.0.0-jar-with-dependencies.jar --countryCode=1 --file=../VietnamNumbers.txt + +# Output: +# Migration of country code +1 phone number(s): +# New numbers file created at: ./+1_Migration_VietnamNumbers.txt +``` +```bash ++1_Migration_VietnamNumbers.txt + +84-1201234567 ++84(121)7654321 ++841225555555 +84 120 1424534 ++84709000000 ++123829734972 +373203934781 +``` +The above migration run created a text file with the same phone numbers as the original file because there are no +1 phone numbers in the file that could have been migrated. + +#### Exporting Invalid Migrations + +When performing standard migrations (migrations that do not use custom recipe files), there is a possibility that a migrated phone number is seen as being invalid for the given country code based on internal checks. This will occur when either an internal recipe is erroneous or the internal phone number ranges for a given country code do not properly reflect all valid numbers. In such cases, migrations will be rolled back and the original number will be written to file when running a “file” migration. To allow for invalid migrations to still be written to file, the “exportInvalidMigrations” flag must be specified. + +The recipes table below shows a recipe with an error which will cause +84120 phone numbers to be converted invalidly to +8460 instead of +8470 which is the actual valid representation for the given Vietnamese numbers: + +```bash +Erroneous Internal Recipe File + +Old Prefix ; Old Length ; Country Code ; Old Format ; New Format ; Description +84120 ; 12 ; 84 ; xx120xxxxxxx ; xx60xxxxxxx ; Redial 84120 with 8470 +84121 ; 12 ; 84 ; xx121xxxxxx ; xx79xxxxxxx ; Redial 84121 with 8479 +``` + +Below is the file input for the example migration runs: + +```bash +VietnamNums.txt + +84-1201234567 ++84(121)7654321 +``` + +##### Example 1 - export original numbers (default): + +```bash +java -jar migrator-1.0.0-jar-with-dependencies.jar --countryCode=84 --file=../VietnamNums.txt + +# Output: +# Migration of country code +84 phone number(s): +# New numbers file created at: ./+84_Migration_VietnamNums.txt +``` + +The created text file will use the original number for the +84120 migration due to the migration being invalid. + +```bash ++84_Migration_VietnamNums.txt + +84-1201234567 ++84797654321 +``` + +##### Example 2 - export invalid migrations: + +```bash +java -jar migrator-1.0.0-jar-with-dependencies.jar --countryCode=84 --file=../VietnamNums.txt --exportInvalidMigrations + +# Output: +# Migration of country code +84 phone number(s): +# New numbers file created at: ./+84_Migration_VietnamNums.txt +``` + +Even though the migration was invalid, the newly created text file will use the migrated number for the +84120 migration because the file is set to be exported with invalid migrations. + +```bash ++84_Migration_VietnamNums.txt + ++84601234567 ++84797654321 +``` + +### Custom Recipe Migrations +Custom recipe migrations can be performed on both “number” and “file” type migrations. The additional requirement is that there is a “customRecipe” argument which points to a csv file containing a recipes table in the standard table format. The following is an example of a valid custom recipe file to be used. + +```bash +CustomRecipes.csv + +Old Prefix ; Old Length ; Country Code ; Old Format ; New Format ; Description +123 ; 7 ; 1 ; xx3xxxx ; xx99xxxx ; Custom recipe, replace 3 with 99 +001234 ; 7 ; 00 ; xxxxxxx ; xx5xxxxx ; Custom recipe, add 5 to third index +``` + +The above recipes file can be used in place of internal recipes by specifying in the following way: + +```bash +java -jar migrator-1.0.0-jar-with-dependencies.jar --countryCode=00 --number=0012345 --customRecipe=./CustomRecipes.csv + +# Output: +# Migration of country code +00 phone number(s): +# Successful migration into: +00512345 +``` +### Creating Custom Recipes.csv Files + +A Recipes.csv file holds a table containing migration information for all known recipes. A given “recipe” is a row within the table which holds data needed to migrate a stale number into one that is valid and dialable. + +When creating custom recipe files to use, there should never be a case where two recipes can be used to renumber one stale number so as a result, a given stale number can only ever match with one row in a Recipes table. Each column in the file must be separated using ```;``` and the table must have the following columns: + +```Prefix (key)``` - the prefix of the possible stale numbers that can be represented by the row. For example, the prefix of the number “12345” could be “12”. Please note that the prefix cannot be the actual number so for the example case, the prefix could not be “12345”. + +```Length (key)``` - the length of all numbers that can be represented in the row. For example a length of 5 for a prefix of “1234” could represent numbers: “12340”, “12341”, “12342”, “12343”, ... + +```Old Format``` - represents what digits in the old format need to change where every digit that needs to stay the same is represented as an ‘x’ +E.g. the number 7 at the 3rd index of a 4 digit number needs to be removed/replaced. This number is always 7 → xx7x +E.g. an addition needs to happen for a 4 digit number and no digits from the original format need to be changed, so none of the original digits need to be altered → xxxx + +```New Format``` - represents what the new number format should be. ‘x’ values represent digits from the original number that will stay the same. +E.g. digits ‘98’ need to be added at the start of an originally 4 digit number → 98xxxx +E.g. no digits need to be added because a removal took place. A 5 digit number has now become a 4 digit number → xxxx + +```Description``` - text specifying what the change is in words. This column is used purely for debugging purposes and will help easily diagnose issues with recipes quickly when the strings in the ‘Old Format’ or ‘New Format’ columns are incorrect. + +## Web Application + +Please [click here](https://phone-number-migrator.uc.r.appspot.com/) to view the web application version of the migrator tool. + +## About validation of phone numbers: + +APIs and data in [libphonenumber/metadata/](https://github.com/google/libphonenumber/tree/master/metadata/) are used for validating phone numbers. More details in [metadata/README](https://github.com/google/libphonenumber/tree/master/metadata/README). + +## Privacy + +Both web application and command line versions of the migrator tool require an input of either a single string E.164 phone number or a text file containing one E.164 phone number per line. After the tool has completed the migration process, this data, along with any resulting data, is never written to any records or stored for later use. + +## Disclaimer + +The migration library is designed to only migrate E.164 phone numbers using internally maintained migration “recipes”. The library will not attempt to migrate national phone numbers or format non-E.164 phone numbers prior to migrations in order for said numbers to be in E.164 format. Only recipes from the specified country code will be used to perform migrations on a given number(s) input. This means that even if a stale E.164 phone number from country A has a recipe that can be used to migrate it successfully but ```--countryCode=B``` is inputted as the command line argument, the phone number will not be migrated. + +For standard migrations using internal recipes, it is possible for E.164 phone numbers to be migrated to a format which the library deems as being invalid. The validity of phone numbers is checked using metadata containing valid number ranges for all countries. As a result, a phone number migration will be seen as invalid if either the metadata of valid number ranges does not reflect the given format correctly or if there is an error with the used recipe internally. In either case, please [file a new issue](https://b.corp.google.com/issues/new?component=192347&template=829703) following the guidance of how to create one [here](https://github.com/google/libphonenumber/blob/master/CONTRIBUTING.md#filing-a-code-issue) in order to resolve the issue. For ```--file``` number inputs, invalid migrations by default will not be written to the created text file containing the updated phone numbers and instead, the original number which correlates to the migration will be used. If you would like to include invalid migration results in the newly created text file, specify this by using the ```--exportInvalidMigrations``` command line flag. + +When using the ```--customRecipe``` argument, all numbers that match a given recipe from the number input will be migrated. These migrations will not be checked for validity based on internal metadata of valid phone number ranges. This means that for custom recipe migrations, there is no perception of invalid migrations. + +## License +See [LICENSE file](https://github.com/google/libphonenumber/blob/master/LICENSE) for Terms and Conditions. diff --git a/migrator/migrator-servlet/pom.xml b/migrator/migrator-servlet/pom.xml new file mode 100644 index 000000000..517d48a4c --- /dev/null +++ b/migrator/migrator-servlet/pom.xml @@ -0,0 +1,94 @@ + + + 4.0.0 + war + 1.0.0 + + com.google.i18n.libphonenumber + migrator-servlet + + + UTF-8 + UTF-8 + 1.8 + 1.8 + true + true + + + + 3.5 + + + + + + com.google.appengine + appengine-api-1.0-sdk + 1.9.82 + + + javax.servlet + javax.servlet-api + 3.1.0 + jar + provided + + + jstl + jstl + 1.2 + + + com.google.i18n.libphonenumber + migrator + 1.0.0-SNAPSHOT + compile + + + javax.servlet.jsp + javax.servlet.jsp-api + 2.3.1 + provided + + + commons-fileupload + commons-fileupload + 1.3.3 + compile + + + + + + + com.google.cloud.tools + appengine-maven-plugin + 1.3.1 + + + + + + org.codehaus.mojo + versions-maven-plugin + 2.3 + + + compile + + display-dependency-updates + display-plugin-updates + + + + + + javax.servlet:javax.servlet-api + com.google.guava:guava + + + + + + diff --git a/migrator/migrator-servlet/src/main/java/com/google/phonenumbers/ServletMain.java b/migrator/migrator-servlet/src/main/java/com/google/phonenumbers/ServletMain.java new file mode 100644 index 000000000..52e75ddd1 --- /dev/null +++ b/migrator/migrator-servlet/src/main/java/com/google/phonenumbers/ServletMain.java @@ -0,0 +1,261 @@ +/* + * Copyright (C) 2020 The Libphonenumber Authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.phonenumbers; + +import com.google.common.base.CharMatcher; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableSet; +import com.google.phonenumbers.migrator.MigrationEntry; +import com.google.phonenumbers.migrator.MigrationFactory; +import com.google.phonenumbers.migrator.MigrationJob; +import com.google.phonenumbers.migrator.MigrationResult; +import org.apache.commons.fileupload.FileItemIterator; +import org.apache.commons.fileupload.FileItemStream; +import org.apache.commons.fileupload.FileUploadException; +import org.apache.commons.fileupload.servlet.ServletFileUpload; +import org.apache.commons.fileupload.util.Streams; +import org.apache.commons.io.IOUtils; + +import javax.servlet.ServletException; +import javax.servlet.annotation.WebServlet; +import javax.servlet.http.HttpServlet; +import javax.servlet.http.HttpServletRequest; +import javax.servlet.http.HttpServletResponse; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.util.StringTokenizer; + +@WebServlet(name = "Migrate", value = "/migrate") +public class ServletMain extends HttpServlet { + + public static final int MAX_UPLOAD_SIZE = 50000; + /** + * Countries with large valid number ranges cannot be migrated using the web application due to request timeouts. The + * command line tool must be used in such cases. + */ + // TODO: add loading spinner and longer request timeouts to UI to allow for migrations of given country codes + public static final ImmutableSet LARGE_COUNTRY_RANGES = ImmutableSet.of( + "1", // US/Canada -- 9.8MB + "86", // China -- 16.1MB + "55", // Brazil -- 3.4MB + "61" // Australia -- 12.8MB + ); + + /** + * Retrieves the form data for either a single number migration or a file migration from the index.jsp file and calls + * the relevant method to perform the migration + */ + @Override + protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException { + ServletFileUpload upload = new ServletFileUpload(); + String countryCode = ""; + String number = ""; + String file = ""; + String fileName = ""; + String customRecipe = ""; + + try { + upload.setSizeMax(MAX_UPLOAD_SIZE); + FileItemIterator iterator = upload.getItemIterator(req); + while (iterator.hasNext()) { + FileItemStream item = iterator.next(); + InputStream in = item.openStream(); + + if (item.isFormField() && (item.getFieldName().equals("numberCountryCode") + || item.getFieldName().equals("fileCountryCode"))) { + countryCode = Streams.asString(in); + + } else if (item.isFormField() && item.getFieldName().equals("number")) { + number = Streams.asString(in); + + } else if (item.getFieldName().equals("file")) { + fileName = item.getName(); + try { + file = IOUtils.toString(in); + } finally { + IOUtils.closeQuietly(in); + } + + } else { + try { + customRecipe = IOUtils.toString(in); + } finally { + IOUtils.closeQuietly(in); + } + } + } + } catch (FileUploadException e) { + e.printStackTrace(); + } + + if (!number.isEmpty() && !countryCode.isEmpty()) { + /* + number and country code are being set again to allow users to see their inputs after the http request has + re-rendered the page. + */ + req.setAttribute("number", number); + req.setAttribute("numberCountryCode", countryCode); + + handleSingleNumberMigration(req, resp, number, countryCode, customRecipe); + + } else if (!file.isEmpty() && !countryCode.isEmpty()) { + ImmutableList.Builder numbersFromFile = ImmutableList.builder(); + StringTokenizer tokenizer = new StringTokenizer(file, "\n"); + while (tokenizer.hasMoreTokens()) { + numbersFromFile.add(tokenizer.nextToken()); + } + + handleFileMigration(req, resp, numbersFromFile.build(), countryCode, customRecipe, fileName); + } + } + + /** Retrieves the form data for the numbers after a file migration and downloads them as a text file. */ + @Override + protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException { + CharMatcher matcher = CharMatcher.anyOf("/\\"); + String fileName = "+" + matcher.removeFrom(req.getParameter("countryCode")) + "_Migration_ " + + matcher.removeFrom(req.getParameter("fileName")); + String fileContent = req.getParameter("fileContent"); + + resp.setContentType("text/plain"); + resp.setHeader("Content-Disposition", "attachment; filename=" + fileName); + try { + OutputStream outputStream = resp.getOutputStream(); + outputStream.write(fileContent.getBytes()); + outputStream.flush(); + outputStream.close(); + } catch (IOException e) { + req.setAttribute("fileError", e.getMessage()); + req.getRequestDispatcher("index.jsp").forward(req, resp); + } + } + + /** + * Performs a single number migration of a given number and country code and sends the details of the migration to the + * jsp file which outputs it to the user. + * + * @throws RuntimeException, which can be any of the given exceptions when going through the process of creating a {@link + * MigrationJob} and running a migration which includes IllegalArgumentExceptions. + */ + public void handleSingleNumberMigration(HttpServletRequest req, + HttpServletResponse resp, + String number, + String countryCode, + String customRecipe) + throws ServletException, IOException { + + if (customRecipe.isEmpty() && LARGE_COUNTRY_RANGES.contains(countryCode)) { + req.setAttribute("numberError", "'+" + countryCode + "' migrations cannot be performed using this web" + + " application, please follow the documentation link above to use the command line tool instead."); + req.getRequestDispatcher("index.jsp").forward(req, resp); + return; + } + + MigrationJob job; + try { + if (!customRecipe.isEmpty()) { + job = MigrationFactory.createCustomRecipeMigration(number, countryCode, MigrationFactory + .importRecipes(IOUtils.toInputStream(customRecipe))); + } else { + job = MigrationFactory.createMigration(number, countryCode); + } + MigrationJob.MigrationReport report = job.getMigrationReportForCountry(); + if (report.getValidMigrations().size() == 1) { + req.setAttribute("validMigration", report.getValidMigrations().get(0).getMigratedNumber()); + } else if (report.getInvalidMigrations().size() == 1) { + req.setAttribute("invalidMigration", report.getInvalidMigrations().get(0).getMigratedNumber()); + } else if (report.getValidUntouchedEntries().size() == 1) { + req.setAttribute("alreadyValidNumber", report.getValidUntouchedEntries().get(0).getSanitizedNumber()); + } + + } catch (RuntimeException e) { + req.setAttribute("numberError", e.getMessage()); + + } finally { + req.getRequestDispatcher("index.jsp").forward(req, resp); + } + } + + /** + * Performs a file migration of a given numbersList and country code and sends the details of the migration to the + * jsp file which outputs it to the user. + * + * @throws RuntimeException, which can be any of the given exceptions when going through the process of creating a {@link + * MigrationJob} and running a migration which includes IllegalArgumentExceptions. + */ + public void handleFileMigration(HttpServletRequest req, HttpServletResponse resp, + ImmutableList numbersList, + String countryCode, + String customRecipe, + String fileName) + throws ServletException, IOException { + + if (customRecipe.isEmpty() && LARGE_COUNTRY_RANGES.contains(countryCode)) { + req.setAttribute("fileError", "'+" + countryCode + "' migrations cannot be performed using this web" + + " application, please follow the documentation link above to use the command line tool instead."); + req.getRequestDispatcher("index.jsp").forward(req, resp); + return; + } + + MigrationJob job; + try { + if (!customRecipe.isEmpty()) { + job = MigrationFactory.createCustomRecipeMigration(numbersList, countryCode, MigrationFactory + .importRecipes(IOUtils.toInputStream(customRecipe))); + } else { + job = MigrationFactory.createMigration(numbersList, countryCode, /* exportInvalidMigrations= */ false); + } + + MigrationJob.MigrationReport report = job.getMigrationReportForCountry(); + // List converted into a Set to allow for constant time contains() method below + ImmutableSet validUntouchedEntriesSet = ImmutableSet.copyOf(report.getValidUntouchedEntries()); + + req.setAttribute("fileName", fileName); + req.setAttribute("fileContent", report.toString()); + req.setAttribute("fileCountryCode", report.getCountryCode()); + req.setAttribute("validMigrations", report.getValidMigrations()); + req.setAttribute("invalidMigrations", report.getInvalidMigrations()); + req.setAttribute("validUntouchedNumbers", report.getValidUntouchedEntries()); + req.setAttribute("invalidUntouchedNumbers", report.getUntouchedEntries().stream() + .filter(entry -> !validUntouchedEntriesSet.contains(entry)).collect(ImmutableList.toImmutableList())); + + } catch (RuntimeException e) { + req.setAttribute("fileError", e.getMessage()); + + } finally { + req.getRequestDispatcher("index.jsp").forward(req, resp); + } + } + + /** + * Takes a list of {@link MigrationEntry}'s and returns a list with the corresponding strings to output for each value + * in the given entryList + */ + public static ImmutableList getMigrationEntryOutputList(ImmutableList entryList) { + if (entryList == null) return ImmutableList.of(); + return entryList.stream().map(MigrationEntry::getOriginalNumber).collect(ImmutableList.toImmutableList()); + } + + /** + * Takes a list of {@link MigrationResult}'s and returns a list with the corresponding strings to output for each value + * in the given resultList + */ + public static ImmutableList getMigrationResultOutputList(ImmutableList resultList) { + if (resultList == null) return ImmutableList.of(); + return resultList.stream().map(MigrationResult::toString).collect(ImmutableList.toImmutableList()); + } +} diff --git a/migrator/migrator-servlet/src/main/webapp/WEB-INF/appengine-web.xml b/migrator/migrator-servlet/src/main/webapp/WEB-INF/appengine-web.xml new file mode 100644 index 000000000..1d7dab28b --- /dev/null +++ b/migrator/migrator-servlet/src/main/webapp/WEB-INF/appengine-web.xml @@ -0,0 +1,8 @@ + + + java8 + true + + + + diff --git a/migrator/migrator-servlet/src/main/webapp/WEB-INF/logging.properties b/migrator/migrator-servlet/src/main/webapp/WEB-INF/logging.properties new file mode 100644 index 000000000..e6e055ecd --- /dev/null +++ b/migrator/migrator-servlet/src/main/webapp/WEB-INF/logging.properties @@ -0,0 +1,4 @@ +# A default java.util.logging configuration. +# (All App Engine logging is through java.util.logging by default). + +.level = WARNING diff --git a/migrator/migrator-servlet/src/main/webapp/WEB-INF/web.xml b/migrator/migrator-servlet/src/main/webapp/WEB-INF/web.xml new file mode 100644 index 000000000..8ddeb6d47 --- /dev/null +++ b/migrator/migrator-servlet/src/main/webapp/WEB-INF/web.xml @@ -0,0 +1,9 @@ + + + + index.jsp + + diff --git a/migrator/migrator-servlet/src/main/webapp/index.jsp b/migrator/migrator-servlet/src/main/webapp/index.jsp new file mode 100644 index 000000000..73205b521 --- /dev/null +++ b/migrator/migrator-servlet/src/main/webapp/index.jsp @@ -0,0 +1,250 @@ +<%@ page import="com.google.appengine.repackaged.com.google.gson.Gson" %> +<%@ page import="com.google.common.collect.ImmutableList" %> +<%@ page import="com.google.phonenumbers.ServletMain" %> +<%@ page import="com.google.phonenumbers.migrator.MigrationEntry" %> +<%@ page import="com.google.phonenumbers.migrator.MigrationResult" %> + +<%@ page contentType="text/html;charset=UTF-8" language="java" %> +<% + final String E164_NUMBERS_LINK = "https://support.twilio.com/hc/en-us/articles/223183008-Formatting-International-Phone-Numbers"; + final String COUNTRY_CODE_LINK = "https://countrycode.org/"; + // TODO: use documentation link from base repository when forked repository has been merged in + final String DOCUMENTATION_LINK = "https://github.com/TomiwaOke/libphonenumber/tree/master/migrator/README.md"; + final String ISSUE_TRACKER_LINK = "https://issuetracker.google.com/issues/new?component=192347"; + final String GUIDELINES_LINK = "https://github.com/google/libphonenumber/blob/master/CONTRIBUTING.md#filing-a-code-issue"; + + final Gson gson = new Gson(); + ImmutableList validMigrations = (ImmutableList) request.getAttribute("validMigrations"); + ImmutableList invalidMigrations = (ImmutableList) request.getAttribute("invalidMigrations"); + ImmutableList validUntouchedNums = (ImmutableList) request.getAttribute("validUntouchedNumbers"); + ImmutableList invalidUntouchedNums = (ImmutableList) request.getAttribute("invalidUntouchedNumbers"); +%> + + + + Migrator + + + + +
+

Phone Number Migrator

+

+ The migrator is a tool which takes in a given E.164 phone number(s) + input as well as the corresponding BCP-47 country code. The tool + will then check the validity of the phone number based on the country code and if possible, will convert the number + into a valid, dialable format. +

+

The following are the two available migration types that can be performed:

+
    +
  • + Single Number Migration: input a single E.164 phone number with its corresponding BCP-47 country + code. If there is an available migration that can be performed on the number, it will be converted to the new + format based on the specified migration rules.

    +
    • +
  • + File Migration: input a text file containing one E.164 number per line along with the BCP-47 + country code that corresponds to the numbers in the text file. All numbers in the text file that match available + migrations will be migrated and there will be the option of downloading a new text file containing the updated numbers. + By default, invalid migrations and numbers that did not go through a process of migration will be written to file + in their original text file format. +

    +
    • +
+

+ For more information on the capabilities of the migrator as well as instructions on how to install the command line + tool, please view the documentation. +

+
+ +
+ <% + if (request.getAttribute("numberError") == null && request.getAttribute("number") != null) { + if (request.getAttribute("validMigration") != null) { + out.print("

Valid +" + request.getAttribute("numberCountryCode") + " Phone Number Produced!

"); + out.print("

The stale number '" + request.getAttribute("number") + "' was successfully migrated into the" + + " phone number: +" + request.getAttribute("validMigration") + "

"); + } else if (request.getAttribute("invalidMigration") != null) { + out.print("

Invalid +" + request.getAttribute("numberCountryCode") + " Migration

"); + out.print("

The stale number '" + request.getAttribute("number") + "' was migrated into the phone number:" + + " +" + request.getAttribute("invalidMigration") + ". However this was not seen as valid using our internal" + + " metadata for country code +" + request.getAttribute("numberCountryCode") + ".

"); + } else if (request.getAttribute("alreadyValidNumber") != null) { + out.print("

Already Valid +" + request.getAttribute("numberCountryCode") + " Phone Number!

"); + out.print("

The entered phone number was already seen as being in a valid, dialable format based on our" + + " metadata for country code +" + request.getAttribute("numberCountryCode") + ". Here is the number in" + + " its clean E.164 format: +" + request.getAttribute("alreadyValidNumber") + "

"); + } else { + out.print("

Non-migratable +" + request.getAttribute("numberCountryCode") + " Phone Number

"); + out.print("

The phone number '" + request.getAttribute("number") + "' was not seen as a valid number and" + + " no migration recipe could be found for country code +" + request.getAttribute("numberCountryCode") + + " to migrate it. This may be because you have entered a country code which does not correctly correspond" + + " to the given phone number or the specified number has never been valid.

"); + } + out.print("

Think there's an issue? File one here following the given guidelines.

"); + } else if (request.getAttribute("fileError") == null && request.getAttribute("fileName") != null) { + out.print("

'" + request.getAttribute("fileName") + "' Migration Report for Country Code: +" + request.getAttribute("fileCountryCode") + "

"); + out.print("

Below is a chart showing the ratio of numbers from the entered file that were able to be migrated" + + " using '+" + request.getAttribute("fileCountryCode") + "' migration recipes. To understand more," + + " select a given segment from the chart below.

"); + out.print("
"); + + out.print("
"); + out.print(""); + out.print(""); + out.print(""); + out.print(""); + out.print("
"); + } + %> +
+ +
+
+

Single Number Migration

+
<%=request.getAttribute("numberError") == null ? "" : request.getAttribute("numberError")%>
+
+ +

Enter a phone number in E.164 format. Inputted numbers can include spaces, curved brackets and hyphens

+ "/> + + +

Enter the BCP-47 country code in which the specified E.164 phone number belongs to

+ "/> + + +

+ (Optional) Upload a csv file containing a custom recipes table to be used for migrations. To understand how to + create a custom recipe file, please view the documentation. +

+ + + +
+
+ +
+

File Migration

+
<%=request.getAttribute("fileError") == null ? "" : request.getAttribute("fileError")%>
+
+ +

Upload a file containing one E.164 phone number per line. Numbers can include spaces, curved brackets and hyphens

+ + + +

Enter the BCP-47 country code in which the E.164 phone numbers from the specified file belong to

+ + + +

+ (Optional) Upload a csv file containing a custom recipes table to be used for migrations. To understand how to + create a custom recipe file, please view the documentation. +

+ + + +
+
+
+ + + + diff --git a/migrator/migrator-servlet/src/main/webapp/stylesheets/servlet-main.css b/migrator/migrator-servlet/src/main/webapp/stylesheets/servlet-main.css new file mode 100644 index 000000000..cd1ae6162 --- /dev/null +++ b/migrator/migrator-servlet/src/main/webapp/stylesheets/servlet-main.css @@ -0,0 +1,138 @@ +.page-heading { + margin: 1rem; + max-width: 75rem; + min-width: 25rem; + font-family: 'Roboto', Arial, sans-serif; +} + +.page-heading h1 { + color: #1a73e8; +} + +.page-heading li { + font-size: 14px; +} + +.migration-forms { + display: flex; + flex-wrap: wrap; +} + +.migration-form { + border: 1px solid #dfdfdf; + padding-left: 1rem; + border-radius: 4px; + margin: 1rem; + max-width: 35rem; + font-family: 'Roboto', Arial, sans-serif; +} + +.migration-form form { + display: flex; + flex-direction: column; + min-width: 25rem; +} + +.migration-form input { + width: 12rem; + margin-bottom: 2rem; +} + +.migration-form p { + font-size: 12px; + color: grey; + width: 95%; +} + +.button { + cursor: pointer; + color: white; + background-color: #1a73e8; + border: none; + padding: 0.5rem; + border-radius: 4px; + font-family: 'Roboto', Arial, sans-serif; +} + +.migration-result { + margin: 1rem; + border-radius: 4px; + border: 1px solid #dfdfdf; + padding: 0 1rem; + max-width: 72rem; + min-width: 25rem; + font-family: 'Roboto', Arial, sans-serif; +} + +.migration-result .valid { + color: green; +} + +.migration-result .invalid-migration { + color: orange; +} + +.migration-result .invalid-number { + color: red; +} + +.error-message { + font-family: 'Roboto', Arial, sans-serif; + word-break: break-word; + margin-top: -0.5rem; + margin-bottom: 1rem; + font-size: 12px; + color: red; + width: 90%; +} + +.chart-wrap { + position: relative; + overflow: hidden; + height: 300px; +} + +.chart { + width: 450px; + height: 300px; + margin: auto; + cursor: pointer; +} + +.modal-backdrop { + display: none; + position: fixed; + z-index: 1; + left: 0; + top: 0; + width: 100%; + height: 100%; + background-color: rgba(0,0,0,0.4); +} + +.modal-content { + font-family: 'Roboto', Arial, sans-serif; + position: relative; + border: 1px solid #dfdfdf; + border-radius: 4px; + background-color: #fefefe; + margin: 2% auto; + padding: 0 1rem 1rem; + width: 20rem; + height: 25rem; +} + +.modal-content .body { + height: 16rem; + overflow: auto; +} + +.modal-content li { + margin-bottom: 0.5rem; +} + +.modal-content button { + position: absolute; + bottom: 1rem; + left: 1rem; +} diff --git a/migrator/pom.xml b/migrator/pom.xml new file mode 100644 index 000000000..ae1104ac1 --- /dev/null +++ b/migrator/pom.xml @@ -0,0 +1,89 @@ + + 4.0.0 + + com.google.i18n.libphonenumber + migrator + 1.0.0-SNAPSHOT + + + + + maven-assembly-plugin + + + + com.google.phonenumbers.migrator.CommandLineMain + + + + jar-with-dependencies + + + + + org.apache.maven.plugins + maven-compiler-plugin + + 8 + 8 + + + + + + ../metadata + + metadata.zip + + + + src/data + + + + + + + com.google.i18n.libphonenumber + metadata + 1.0-SNAPSHOT + compile + + + junit + junit + 4.13 + test + + + com.google.truth + truth + 1.0.1 + test + + + info.picocli + picocli + 4.5.0 + + + com.google.truth.extensions + truth-java8-extension + 1.0.1 + test + + + com.google.auto.value + auto-value + 1.7.4 + provided + + + com.google.auto.value + auto-value-annotations + 1.7.4 + + + diff --git a/migrator/src/data/recipes.csv b/migrator/src/data/recipes.csv new file mode 100644 index 000000000..8be31c037 --- /dev/null +++ b/migrator/src/data/recipes.csv @@ -0,0 +1,185 @@ +Old Prefix ; Old Length ; Country Code ; Old Format ; New Format ; Is Final Migration ; Description +8412[068] ; 12 ; 84 ; xx12xxxxxxxx ; xx7xxxxxxxx ; true ; redial +8412(0|6|8) with +847(0|6|8) +84121 ; 12 ; 84 ; xx121xxxxxxx ; xx79xxxxxxx ; true ; redial +84121 with +8479 +84122 ; 12 ; 84 ; xx122xxxxxxx ; xx77xxxxxxx ; true ; redial +84122 with +8477 +8412[3-5] ; 12 ; 84 ; xx12xxxxxxxx ; xx8xxxxxxxx ; true ; redial +8412(3|4|5) with +848(3|4|5) +84127 ; 12 ; 84 ; xx127xxxxxxx ; xx81xxxxxxx ; true ; redial +84127 with +8481 +84129 ; 12 ; 84 ; xx129xxxxxxx ; xx82xxxxxxx ; true ; redial +84129 with +8482 +8416[2-9] ; 12 ; 84 ; xx16xxxxxxxx ; xx3xxxxxxxx ; true ; redial +8416(2|3|4|5|6|7|8|9) with +843(2|3|4|5|6|7|8|9) +8418[68] ; 12 ; 84 ; xx18xxxxxxxx ; xx5xxxxxxxx ; true ; redial +8418(6|8) with +845(6|8) +84199 ; 12 ; 84 ; xx199xxxxxxx ; xx59xxxxxxx ; true ; redial +84199 with +8459 +84992 ; 11 ; 84 ; xx992xxxxxx ; xx672xxxxxx ; true ; redial +84992 with +84672 +847[235] ; 11 ; 84 ; xx7xxxxxxxx ; xx27xxxxxxxx ; true ; redial +847(2|3|5) with +8427(2|3|5) +8468 ; 11 ; 84 ; xx68xxxxxxx ; xx259xxxxxxx ; true ; redial +8468 with +84259 +8467[013-9] ; 11 ; 84 ; xx6xxxxxxxx ; xx27xxxxxxxx ; true ; redial +8467 with +84277 +8466 ; 11 ; 84 ; xx6xxxxxxxx ; xx27xxxxxxxx ; true ; redial +8466 with +84276 +84651 ; 12 ; 84 ; xx651xxxxxxx ; xx271xxxxxxx ; true ; redial +84651 with +84271 +84650 ; 12 ; 84 ; xx650xxxxxxx ; xx274xxxxxxx ; true ; redial +84650 with +84274 +846[124] ; 11 ; 84 ; xx6xxxxxxxx ; xx25xxxxxxxx ; true ; redial +846(1|2|4) with +8425(1|2|4) +846[03] ; 11 ; 84 ; xx6xxxxxxxx ; xx26xxxxxxxx ; true ; redial +846(0|3) with +8426(0|3) +8457 ; 11 ; 84 ; xx5xxxxxxxx ; xx25xxxxxxxx ; true ; redial +8457 with +84257 +8455[0-8] ; 11 ; 84 ; xx5xxxxxxxx ; xx25xxxxxxxx ; true ; redial +8455 with +84255 +84501 ; 12 ; 84 ; xx501xxxxxxx ; xx261xxxxxxx ; true ; redial +84501 with +84261 +84500 ; 12 ; 84 ; xx500xxxxxxx ; xx262xxxxxxx ; true ; redial +84500 with +84262 +844[2-8] ; 11 ; 84 ; xx4xxxxxxxx ; xx24xxxxxxxx ; true ; redial +844 with +8424 +8474 ; 11 ; 84 ; xx7xxxxxxxx ; xx29xxxxxxxx ; true ; redial +8474 with +84294 +84711 ; 12 ; 84 ; xx711xxxxxxx ; xx293xxxxxxx ; true ; redial +84711 with +84293 +84710 ; 12 ; 84 ; xx710xxxxxxx ; xx292xxxxxxx ; true ; redial +84710 with +84292 +84280 ; 12 ; 84 ; xx280xxxxxxx ; xx208xxxxxxx ; true ; redial +84280 with +84208 +842[5-7] ; 11 ; 84 ; xx2xxxxxxxx ; xx20xxxxxxxx ; true ; redial +842 with +8420 +84240 ; 12 ; 84 ; xx240xxxxxxx ; xx204xxxxxxx ; true ; redial +84240 with +84204 +845[34] ; 11 ; 84 ; xx5xxxxxxxx ; xx23xxxxxxxx ; true ; redial +845 with +8423 +8452[014-79] ; 11 ; 84 ; xx5xxxxxxxx ; xx23xxxxxxxx ; true ; redial +845 with +8423 +84511 ; 12 ; 84 ; xx511xxxxxxx ; xx236xxxxxxx ; true ; redial +84511 with +84236 +84510 ; 12 ; 84 ; xx510xxxxxxx ; xx235xxxxxxx ; true ; redial +84510 with +84235 +8429 ; 11 ; 84 ; xx29xxxxxxx ; xx216xxxxxxx ; true ; redial +8429 with +84216 +84231 ; 12 ; 84 ; xx231xxxxxxx ; xx213xxxxxxx ; true ; redial +84231 with +84213 +84230 ; 12 ; 84 ; xx230xxxxxxx ; xx215xxxxxxx ; true ; redial +84230 with +84215 +8422 ; 11 ; 84 ; xx22xxxxxxx ; xx212xxxxxxx ; true ; redial +8422 with +84212 +8420 ; 11 ; 84 ; xx20xxxxxxx ; xx214xxxxxxx ; true ; redial +8420 with +84214 +63220[89] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63220 with +632820 +63221 ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63221 with +632721 +63222[0-46-8] ; 10 ; 63 ; xxxxxxxxxx ; xxx3xxxxxxx ; true ; redial +63222 with +632322 +632225 ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +632225 with +6327225 +63223[0-7] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63223 with +632823 +63223[89] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63223(8|9) with +632723(8|9) +6322[45] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +6322(4|5) with +63282(4|5) +63226[07-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63226 with +632826 +63226[13-6] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63226(1|3-6) with +632726(1|3-6) +632262 ; 10 ; 63 ; xxxxxxxxxx ; xxx3xxxxxxx ; true ; redial +632262 with +6323262 +6322[7-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +6322(7|8|9) with +63282(7|8|9) +63233[0-4] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63233 with +632833 +63234[0-8] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63234 with +632834 +63235[0-79] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63235 with +632835 +632358 ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +632358 with +6328358 +63236[0-7] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63236 with +632836 +63236[89] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63236(8|9) with +632736(8|9) +63237[0-6] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63237 with +632837 +63237[7-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx3xxxxxxx ; true ; redial +63237(7|8|9) with +632337(7|8|9) +63238 ; 10 ; 63 ; xxxxxxxxxx ; xxx3xxxxxxx ; true ; redial +63238 with +632338 +63239[0-4] ; 10 ; 63 ; xxxxxxxxxx ; xxx3xxxxxxx ; true ; redial +63239(0-4) with +632339(0-4) +63239[5-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63239(5-9) with +632839(5-9) +63240[0-5] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63240(0-5) with +632840(0-5) +63240[6-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx3xxxxxxx ; true ; redial +63240(6-9) with +632340(6-9) +6324[139] ; 10 ; 63 ; xxxxxxxxxx ; xxx3xxxxxxx ; true ; redial +6324(1|3|9) with +63234(1|3|9) +63242[0-3569] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63242 with +632842 +63242[78] ; 10 ; 63 ; xxxxxxxxxx ; xxx3xxxxxxx ; true ; redial +63242(7|8) with +632342(7|8) +63244[03-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx3xxxxxxx ; true ; redial +63244 with +632344 +63244[12] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63244(1|2) with +632844(1|2) +63245[03-6] ; 10 ; 63 ; xxxxxxxxxx ; xxx3xxxxxxx ; true ; redial +63245 with +632345 +63245[127-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63245(1|2|7-9) with +632845(1|2|7-9) +63246[0-5] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63246(0-5) with +632846(0-5) +63246[6-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx3xxxxxxx ; true ; redial +63246(6-9) with +632346(6-9) +63247[0-25-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63247 with +632847 +632473 ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +632473 with +6327473 +632474 ; 10 ; 63 ; xxxxxxxxxx ; xxx3xxxxxxx ; true ; redial +632474 with +6323474 +63248[013-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx3xxxxxxx ; true ; redial +63248 with +632348 +632482 ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +632482 with +6327482 +63250 ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63250 with +632750 +6325[1256] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +6325(1|2|5|6) with +63285(1|2|5|6) +63253[0-6] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63253(0-6) with +632853(0-6) +63254[1-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63254(1-9) with +632854(1-9) +63257[0-589] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63257 with +632857 +63257[67] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63257(6|7) with +632757(6|7) +63258[0-489] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63258 with +632858 +63258[5-7] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63258(5|6|7) with +632758(5|6|7) +63261[135-8] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63261 with +632761 +63262[06-8] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63262 with +632862 +63262[1-5] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63262(1-5) with +632762(1-5) +6326[35] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +6326(3|5) with +63286(3|5) +63264[0-7] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63264 with +632864 +63266[0-46-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63266 with +632866 +63267[12] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63267 with +632867 +63268[1-37-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63268 with +632868 +63269[1-35-7] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63269 with +632869 +63270 ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63270 with +632870 +63271[0-6] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63271 with +632871 +63271[7-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63271(7|8|9) with +632771(7|8|9) +6327[23][089] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +6327(2|3)(0|8|9) with +63277(2|3)(0|8|9) +6327[23][1-7] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +6327(2|3) with +63287(2|3) +63274[0-39] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63274 with +632874 +63274[4-8] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63274(4-8) with +632774(4-8) +63275 ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63275 with +632775 +63276 ; 10 ; 63 ; xxxxxxxxxx ; xxx6xxxxxxx ; true ; redial +63276 with +632676 +6327[78] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +6327(7|8) with +63287(7|8) +632790 ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +632790 with +6328790 +63279[1-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63279(1-9) with +632779(1-9) +6328 ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +6328 with +63288 +63290 ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63290 with +632790 +63291[04-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63291 with +632791 +63291[1-3] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63291(1-3) with +632891(1-3) +63292 ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63292 with +632892 +63293[0-25-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63293 with +632893 +63293[34] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63293(3|4) with +632793(3|4) +63294[03-69] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63294 with +632794 +63294[1278] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63294(1|2|7|8) with +632894(1|2|7|8) +63295[04-9] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63295 with +632795 +63295[1-3] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63295(1-3) with +632895(1-3) +63296[0468] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63296 with +632796 +63296[127] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63296(1|2|7) with +632896(1|2|7) +63297[0-568] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63297 with +632797 +63298[079] ; 10 ; 63 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +63298(0|7|9) with +632798(0|7|9) +63298[1-68] ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63298 with +632898 +63299 ; 10 ; 63 ; xxxxxxxxxx ; xxx8xxxxxxx ; true ; redial +63299 with +632899 +559[1-9][689] ; 12 ; 55 ; xxxxxxxxxxxx ; xxxx9xxxxxxxx ; true ; redial +559(1-9)(6|8|9) with +559(1-9)9(6|8|9) +558[1-9][689] ; 12 ; 55 ; xxxxxxxxxxxx ; xxxx9xxxxxxxx ; true ; redial +558(1-9)(6|8|9) with +558(1-9)9(6|8|9) +553[1-578][689] ; 12 ; 55 ; xxxxxxxxxxxx ; xxxx9xxxxxxxx ; true ; redial +553(1-5|7|8)(6|8|9) with +553(1-5|7|8)9(6|8|9) +557[13-579][689] ; 12 ; 55 ; xxxxxxxxxxxx ; xxxx9xxxxxxxx ; true ; redial +557(1|3-5|7|9) with +557(1|3-5|7|9)9 +556[1-9][689] ; 12 ; 55 ; xxxxxxxxxxxx ; xxxx9xxxxxxxx ; true ; redial +556(1-9) with +556(1-9)9 +554[1-9][689] ; 12 ; 55 ; xxxxxxxxxxxx ; xxxx9xxxxxxxx ; true ; redial +554(1-9) with +554(1-9)9 +555[13-5][689] ; 12 ; 55 ; xxxxxxxxxxxx ; xxxx9xxxxxxxx ; true ; redial +555(1|3-5) with +555(1|3-5)9 +24101 ; 11 ; 241 ; xxx0xxxxxxx ; xxx1xxxxxxx ; true ; redial +24101 with +24111 +2410[256] ; 11 ; 241 ; xxx0xxxxxxx ; xxx6xxxxxxx ; true ; redial +2410(2|5|6) with +2416(2|5|6) +2410[47] ; 11 ; 241 ; xxx0xxxxxxx ; xxx7xxxxxxx ; true ; redial +2410(4|7) with +2417(4|7) +241[256] ; 10 ; 241 ; xxxxxxxxxx ; xxx6xxxxxxx ; true ; redial +241(2|5|6) with +2416(2|5|6) +241[47] ; 10 ; 241 ; xxxxxxxxxx ; xxx7xxxxxxx ; true ; redial +241(4|7) with +2417(4|7) +52122[1-9] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52122 with +5222 +52123[1-35-8] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52123 with +5223 +52124[13-9] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52124 with +5224 +52127[1-689] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52127 with +5227 +52128[1-578] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52128 with +5228 +52129[467] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52129 with +5229 +52131[1-79] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52131 with +5231 +5213[2458][1-9] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +5213(2|4|5|8) with +523(2|4|5|8) +52133 ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52133 with +5233 +52137[1-8] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52137 with +5237 +52139[1-5] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52139 with +5239 +52141[1-57-9] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52141 with +5241 +5214[24-7][1-9] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +5214(2|4-7) with +524(2|4-7) +52143[1-8] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52143 with +5243 +52148[1-35-9] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52148 with +5248 +52149[2-689] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52149 with +5249 +5215[56] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +5215(5|6) with +525(5|6) +521588 ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +521588 with +52588 +52159[1-79] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52159 with +5259 +52161[2-68] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52161 with +5261 +5216[2-4][1-9] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +5216(2-4) with +526(2-4) +52165[1-3689] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52165 with +5265 +52166[1-57-9] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52166 with +5266 +52167[1-7] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52167 with +5267 +52168[67] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52168(6|7) with +5268(6|7) +52169[4-8] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52169 with +5269 +5217[1-467][1-9] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +5217 with +527 +52175[13-9] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52175 with +5275 +52178[1-69] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52178 with +5278 +52179[17] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52179 with +5279 +52181 ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52181 with +5281 +52182[13-689] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52182 with +5282 +52183[1-6] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52183 with +5283 +52184[124-6] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52184 with +5284 +52186[1246-9] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52186 with +5286 +52187[1-378] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52187 with +5287 +52189[12479] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52189 with +5289 +52191[346-9] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52191 with +5291 +52192[1-4] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52192 with +5292 +52193[2-46-8] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52193 with +5293 +52195[1348] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52195 with +5295 +5219[69][1-9] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +5219(6|9) with +529(6|9) +52197[12] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52197 with +5297 +52198[1-8] ; 13 ; 52 ; xx1xxxxxxxxxx ; xxxxxxxxxxxx ; true ; redial +52198 with +5298 +9661 ; 11 ; 966 ; xxxxxxxxxxx ; xxx1xxxxxxxx ; true ; redial +9661 with +96611 +9662[24-8] ; 11 ; 966 ; xxxxxxxxxxx ; xxx1xxxxxxxx ; true ; redial +9662 with +96612 +9663[35-8] ; 11 ; 966 ; xxxxxxxxxxx ; xxx1xxxxxxxx ; true ; redial +9663 with +96613 +9664[3-68] ; 11 ; 966 ; xxxxxxxxxxx ; xxx1xxxxxxxx ; true ; redial +9664 with +96614 +9666[2-5] ; 11 ; 966 ; xxxxxxxxxxx ; xxx1xxxxxxxx ; true ; redial +9666 with +96616 +9667[235-7] ; 11 ; 966 ; xxxxxxxxxxx ; xxx1xxxxxxxx ; true ; redial +9667 with +96617 diff --git a/migrator/src/main/java/com/google/phonenumbers/migrator/CommandLineMain.java b/migrator/src/main/java/com/google/phonenumbers/migrator/CommandLineMain.java new file mode 100644 index 000000000..c2d94b4d2 --- /dev/null +++ b/migrator/src/main/java/com/google/phonenumbers/migrator/CommandLineMain.java @@ -0,0 +1,173 @@ +/* + * Copyright (C) 2020 The Libphonenumber Authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.phonenumbers.migrator; + +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.Multimap; +import com.google.i18n.phonenumbers.metadata.table.Column; +import com.google.phonenumbers.migrator.MigrationJob.MigrationReport; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.Scanner; +import picocli.CommandLine; +import picocli.CommandLine.ArgGroup; +import picocli.CommandLine.Command; +import picocli.CommandLine.Help.Ansi; +import picocli.CommandLine.Option; + +@Command(name = "Command Line Migrator Tool:", + description = "Please enter a path to a text file containing E.164 phone numbers " + + "(e.g. +4434567891, +1234568890) from the same country or a single E.164 number as " + + "well as the corresponding BCP-47 country code (e.g. 44, 1) to begin migrations.\n") +public final class CommandLineMain { + /** + * Fields cannot be private or final to allow for @Command annotation to set and retrieve values. + */ + + @ArgGroup(multiplicity = "1") + NumberInputType numberInput; + + static class NumberInputType { + @Option(names = {"-n", "--number"}, + description = "Single E.164 phone number to migrate (e.g. \"+1234567890\") or an internationally formatted " + + "number starting with '+' (e.g. \"+12 (345) 67-890\") which will have any non-digit characters after " + + "the leading '+' removed for processing.") + String number; + + @Option(names = {"-f", "--file"}, + description = "Text file to be migrated which contains one E.164 phone " + + "number per line") + String file; + } + + @Option(names = {"-c", "--countryCode"}, + required = true, + description = "The BCP-47 country code the given phone number(s) belong to (e.g. 44)") + String countryCode; + + @ArgGroup() + OptionalParameterType optionalParameter; + + static class OptionalParameterType { + @Option(names = {"-e", "--exportInvalidMigrations"}, + description = "boolean flag specifying that text files created after the migration process" + + " for standard recipe --file migrations should contain the migrated version of a given" + + " phone number, regardless of whether the migration resulted in an invalid phone number." + + " By default, a strict approach is used and when a migration is seen as invalid, the" + + " original phone number is written to file. Invalid numbers will be printed at the" + + " bottom of the text file.") + boolean exportInvalidMigrations; + + @Option(names = {"-r", "--customRecipe"}, + description = "Csv file containing a custom migration recipes table. When using custom recipes" + + ", validity checks on migrated numbers will not be performed. Note: custom recipes must" + + " be run with the --exportInvalidMigrations flag.") + String customRecipe; + } + + @Option(names = {"-h", "--help"}, description = "Display help", usageHelp = true) + boolean help; + + public static void main(String[] args) throws IOException { + CommandLineMain clm = CommandLine.populateCommand(new CommandLineMain(), args); + if (clm.help) { + CommandLine.usage(clm, System.out, Ansi.AUTO); + } else { + MigrationJob migrationJob; + if (clm.numberInput.number != null) { + if (clm.optionalParameter != null && clm.optionalParameter.customRecipe != null) { + migrationJob = MigrationFactory + .createCustomRecipeMigration(clm.numberInput.number, clm.countryCode, + MigrationFactory.importRecipes(Files.newInputStream(Paths.get(clm.optionalParameter.customRecipe)))); + } else { + migrationJob = MigrationFactory.createMigration(clm.numberInput.number, clm.countryCode); + } + } else { + if (clm.optionalParameter != null && clm.optionalParameter.customRecipe != null) { + migrationJob = MigrationFactory + .createCustomRecipeMigration(Paths.get(clm.numberInput.file), clm.countryCode, + MigrationFactory.importRecipes(Files.newInputStream(Paths.get(clm.optionalParameter.customRecipe)))); + } else { + migrationJob = MigrationFactory + .createMigration(Paths.get(clm.numberInput.file), clm.countryCode, + clm.optionalParameter != null && clm.optionalParameter.exportInvalidMigrations); + } + } + + MigrationReport mr = migrationJob.getMigrationReportForCountry(); + System.out.println("Migration of country code +" + migrationJob.getCountryCode() + " phone " + + "number(s):"); + if (clm.numberInput.file != null) { + printFileReport(mr, Paths.get(clm.numberInput.file)); + } else { + printNumberReport(mr); + } + } + } + + /** Details printed to console after a --file type migration. */ + private static void printFileReport(MigrationReport mr, Path originalFile) throws IOException { + String newFile = mr.exportToFile(originalFile.getFileName().toString()); + Scanner scanner = new Scanner((System.in)); + String response = ""; + + System.out.println("New numbers file created at: " + System.getProperty("user.dir") + "/" + newFile); + while (!response.equals("0")) { + System.out.println("\n(0) Exit"); + System.out.println("(1) Print Metrics"); + System.out.println("(2) View All Recipes Used"); + System.out.print("Select from the above options: "); + response = scanner.nextLine(); + if (response.equals("1")) { + mr.printMetrics(); + } else if (response.equals("2")) { + printUsedRecipes(mr); + } + } + } + + /** Details printed to console after a --number type migration. */ + private static void printNumberReport(MigrationReport mr) { + if (mr.getValidMigrations().size() == 1) { + System.out.println("Successful migration into: +" + + mr.getValidMigrations().get(0).getMigratedNumber()); + printUsedRecipes(mr); + } else if (mr.getInvalidMigrations().size() == 1) { + System.out.println("The number was migrated into '+" + + mr.getInvalidMigrations().get(0).getMigratedNumber() + "' but this number was not " + + "seen as being valid and dialable when inspecting our data for the given country"); + printUsedRecipes(mr); + } else if (mr.getValidUntouchedEntries().size() == 1) { + System.out.println("This number was seen to already be valid and dialable based on " + + "our data for the given country"); + } else { + System.out.println("This number could not be migrated using any of the recipes from the given" + + " recipes file"); + } + } + + private static void printUsedRecipes(MigrationReport mr) { + Multimap, Object>, MigrationResult> recipeToNumbers = mr.getAllRecipesUsed(); + System.out.println("\nRecipe(s) Used:"); + for (ImmutableMap, Object> recipe : recipeToNumbers.keySet()) { + System.out.println("* " + RecipesTableSchema.formatRecipe(recipe)); + recipeToNumbers.get(recipe).forEach(result -> System.out.println("\t" + result)); + System.out.println(""); + } + } +} diff --git a/migrator/src/main/java/com/google/phonenumbers/migrator/MetadataZipFileReader.java b/migrator/src/main/java/com/google/phonenumbers/migrator/MetadataZipFileReader.java new file mode 100644 index 000000000..b8ed4bcba --- /dev/null +++ b/migrator/src/main/java/com/google/phonenumbers/migrator/MetadataZipFileReader.java @@ -0,0 +1,106 @@ +/* + * Copyright (C) 2020 The Libphonenumber Authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.phonenumbers.migrator; + +import static com.google.common.base.Preconditions.checkNotNull; + +import com.google.i18n.phonenumbers.metadata.DigitSequence; +import com.google.i18n.phonenumbers.metadata.model.RangesTableSchema; +import com.google.i18n.phonenumbers.metadata.table.CsvTable; +import com.google.i18n.phonenumbers.metadata.table.RangeKey; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.nio.file.Files; +import java.nio.file.Paths; +import java.util.Optional; +import java.util.Scanner; +import java.util.zip.ZipEntry; +import java.util.zip.ZipInputStream; + +/** + * Represents a standard libphonenumber metadata zip file where each {@link MetadataZipFileReader} + * zip file contains {@link CsvTable}'s that can be imported based on specified BCP-47 format + * numerical country codes without the need to extract the whole zip file. + */ +public final class MetadataZipFileReader { + + private final ZipInputStream metadataZipFile; + + private MetadataZipFileReader(ZipInputStream metadataZipFile) { + this.metadataZipFile = checkNotNull(metadataZipFile); + } + + /** + * Returns a MetadataZipFileReader for the given zip stream. + */ + public static MetadataZipFileReader of(InputStream file) throws IOException { + checkNotNull(file); + + return new MetadataZipFileReader(new ZipInputStream(file)); + } + + /** + * Returns the {@link CsvTable} for the given BCP-47 numerical country code (e.g. "44") if present. + */ + public Optional> importCsvTable(DigitSequence countryCode) throws IOException { + String csvTableLocation = "metadata/" + countryCode + "/ranges.csv"; + ZipEntry entry; + + while ((entry = metadataZipFile.getNextEntry()) != null) { + if (entry.getName().equals(csvTableLocation)) { + return Optional.of(CsvTable.importCsv(RangesTableSchema.SCHEMA, + new InputStreamReader(metadataZipFile))); + } + } + return Optional.empty(); + } + + /** + * Method used as an example of how to instantiate a MetadataReader object for a given zip file + * and import the {@link CsvTable} corresponding to a given BCP-47 numerical country code. + * + * @param args which expects two command line arguments; + * fileLocation: the path of the zipfile to be read + * countryCode: numerical code relating to the CSV table that is to be imported + */ + public static void main(String[] args) throws IOException, RuntimeException { + String fileLocation; + String countryCode; + + if (args.length < 2) { + Scanner scanner = new Scanner(System.in); + System.out.println("You have not entered correct command line arguments"); + + System.out.print("\tPlease enter a zip file location of metadata csv files: "); + fileLocation = scanner.next(); + + System.out.print("\tPlease enter the country code of the ranges you want to import: "); + countryCode = scanner.next(); + } else { + fileLocation = args[0]; + countryCode = args[1]; + } + + MetadataZipFileReader m = MetadataZipFileReader.of(Files.newInputStream(Paths.get(fileLocation))); + Optional> ranges = Optional.ofNullable( + m.importCsvTable(DigitSequence.of(countryCode)) + .orElseThrow(() -> new RuntimeException("Country code not supported in zipfile"))); + + System.out.println("Table imported!"); + ranges.get().getKeys().forEach(System.out::println); + } +} diff --git a/migrator/src/main/java/com/google/phonenumbers/migrator/MigrationEntry.java b/migrator/src/main/java/com/google/phonenumbers/migrator/MigrationEntry.java new file mode 100644 index 000000000..205138116 --- /dev/null +++ b/migrator/src/main/java/com/google/phonenumbers/migrator/MigrationEntry.java @@ -0,0 +1,33 @@ +/* + * Copyright (C) 2020 The Libphonenumber Authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.phonenumbers.migrator; + +import com.google.auto.value.AutoValue; +import com.google.i18n.phonenumbers.metadata.DigitSequence; + +/** + * Representation of each number to be migrated by a given MigrationJob. Contains the original number + * string as well as the sanitized E.164 {@link DigitSequence}. + */ +@AutoValue +public abstract class MigrationEntry { + public abstract DigitSequence getSanitizedNumber(); + public abstract String getOriginalNumber(); + + public static MigrationEntry create(DigitSequence sanitizedNumber, String originalNumber) { + return new AutoValue_MigrationEntry(sanitizedNumber, originalNumber); + } +} diff --git a/migrator/src/main/java/com/google/phonenumbers/migrator/MigrationFactory.java b/migrator/src/main/java/com/google/phonenumbers/migrator/MigrationFactory.java new file mode 100644 index 000000000..3130671ae --- /dev/null +++ b/migrator/src/main/java/com/google/phonenumbers/migrator/MigrationFactory.java @@ -0,0 +1,177 @@ +/* + * Copyright (C) 2020 The Libphonenumber Authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.phonenumbers.migrator; + +import com.google.common.base.CharMatcher; +import com.google.common.collect.ImmutableList; +import com.google.i18n.phonenumbers.metadata.DigitSequence; +import com.google.i18n.phonenumbers.metadata.table.CsvTable; +import com.google.i18n.phonenumbers.metadata.table.RangeKey; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; + +/** + * Factory class to instantiate {@link MigrationJob} objects. To create a Migration Job, a BCP-47 + * country code is required as well as a number input (either a single E.164 number or a file + * path containing one E.164 number per line). There is also the option for specifying a custom + * recipes file as a third parameter to use for migrations instead of the default recipes file. + */ +public class MigrationFactory { + + // made public for testing purposes + public final static String DEFAULT_RECIPES_FILE = "/recipes.csv"; + public final static String METADATA_ZIPFILE ="/metadata.zip"; + + /** + * Creates a new MigrationJob for a given single E.164 number input (e.g. +4477...) and its + * corresponding BCP-47 country code (e.g. 44 for United Kingdom). + */ + public static MigrationJob createMigration(String number, String country) throws IOException { + DigitSequence countryCode = DigitSequence.of(country); + ImmutableList numberRanges = + ImmutableList.of(MigrationEntry.create(sanitizeNumberString(number), number)); + CsvTable recipes = importRecipes(MigrationFactory.class + .getResourceAsStream(DEFAULT_RECIPES_FILE)); + + MetadataZipFileReader metadata = MetadataZipFileReader.of(MigrationFactory.class + .getResourceAsStream(METADATA_ZIPFILE)); + CsvTable ranges = metadata.importCsvTable(countryCode) + .orElseThrow(() -> new RuntimeException( + "Country code " + countryCode+ " not supported in metadata")); + + return new MigrationJob(numberRanges, countryCode, recipes, + ranges, /* exportInvalidMigrations= */false); + } + + /** + * Returns a MigrationJob instance for a given single E.164 number input, corresponding BCP-47 + * country code (e.g. 1 for Canada), and custom user recipes.csv file. + */ + public static MigrationJob createCustomRecipeMigration(String number, + String country, + CsvTable recipes) { + DigitSequence countryCode = DigitSequence.of(country); + ImmutableList numberRanges = + ImmutableList.of(MigrationEntry.create(sanitizeNumberString(number), number)); + + return new MigrationJob(numberRanges, countryCode, recipes, + /* rangesTable= */null, /* exportInvalidMigrations= */false); + } + + /** + * Returns a MigrationJob instance for a given file path containing one E.164 + * number per line (e.g. +4477..., +4478...) along with the corresponding + * BCP-47 country code (e.g. 44) that numbers in the file belong to. + * All numbers in the file should belong to the same region. + */ + public static MigrationJob createMigration(Path file, String country, boolean exportInvalidMigrations) + throws IOException { + List numbers = Files.readAllLines(file); + DigitSequence countryCode = DigitSequence.of(country); + ImmutableList.Builder numberRanges = ImmutableList.builder(); + + numbers.forEach(num -> numberRanges.add(MigrationEntry.create(sanitizeNumberString(num), num))); + CsvTable recipes = importRecipes(MigrationFactory.class + .getResourceAsStream(DEFAULT_RECIPES_FILE)); + + MetadataZipFileReader metadata = MetadataZipFileReader.of(MigrationFactory.class + .getResourceAsStream(METADATA_ZIPFILE)); + CsvTable ranges = metadata.importCsvTable(countryCode) + .orElseThrow(() -> new RuntimeException( + "Country code " + countryCode+ " not supported in metadata")); + + return new MigrationJob(numberRanges.build(), countryCode, recipes, ranges, exportInvalidMigrations); + } + + /** + * Returns a MigrationJob instance for a given file path containing one E.164 + * number per line, corresponding BCP-47 country code, and custom user recipes.csv file. + */ + public static MigrationJob createCustomRecipeMigration(Path file, + String country, + CsvTable recipes) + throws IOException { + List numbers = Files.readAllLines(file); + DigitSequence countryCode = DigitSequence.of(country); + ImmutableList.Builder numberRanges = ImmutableList.builder(); + + numbers.forEach(num -> numberRanges.add(MigrationEntry.create(sanitizeNumberString(num), num))); + + return new MigrationJob(numberRanges.build(), countryCode, recipes, + /* rangesTable= */null, /* exportInvalidMigrations= */ false); + } + + /** + * Returns a MigrationJob instance for a given file path containing one E.164 number per line, corresponding BCP-47 + * country code, and custom user recipes.csv file. Method needed for migrations using migrator-servlet. + */ + public static MigrationJob createCustomRecipeMigration(ImmutableList numbers, + String country, + CsvTable recipes) throws IOException { + DigitSequence countryCode = DigitSequence.of(country); + ImmutableList.Builder numberRanges = ImmutableList.builder(); + numbers.forEach(num -> numberRanges.add(MigrationEntry.create(sanitizeNumberString(num), num))); + + return new MigrationJob(numberRanges.build(), countryCode, recipes, + /* rangesTable= */null, /* exportInvalidMigrations= */ false); + } + + /** + * Returns a MigrationJob instance for a given list of E.164 numbers. Method needed for file migrations using migrator + * servlet. + */ + public static MigrationJob createMigration(ImmutableList numbers, String country, boolean exportInvalidMigrations) + throws IOException { + DigitSequence countryCode = DigitSequence.of(country); + ImmutableList.Builder numberRanges = ImmutableList.builder(); + + numbers.forEach(num -> numberRanges.add(MigrationEntry.create(sanitizeNumberString(num), num))); + CsvTable recipes = importRecipes(MigrationFactory.class + .getResourceAsStream(DEFAULT_RECIPES_FILE)); + + MetadataZipFileReader metadata = MetadataZipFileReader.of(MigrationFactory.class + .getResourceAsStream(METADATA_ZIPFILE)); + CsvTable ranges = metadata.importCsvTable(countryCode) + .orElseThrow(() -> new RuntimeException( + "Country code " + countryCode+ " not supported in metadata")); + + return new MigrationJob(numberRanges.build(), countryCode, recipes, ranges, exportInvalidMigrations); + } + + /** + * Removes spaces and '+' '(' ')' '-' characters expected in E.164 numbers then returns the + * {@link DigitSequence} representation of a given number. The method will not remove other + * letters or special characters from strings to enable error messages in cases where invalid + * numbers are inputted. + */ + private static DigitSequence sanitizeNumberString(String number) { + CharMatcher matcher = CharMatcher.anyOf("-+()").or(CharMatcher.whitespace()); + return DigitSequence.of(matcher.removeFrom(number)); + } + + /** + * Returns the {@link CsvTable} for a given recipes file path if present. + * Made public for testing purposes. + */ + public static CsvTable importRecipes(InputStream recipesFile) throws IOException { + InputStreamReader reader = new InputStreamReader(recipesFile); + return CsvTable.importCsv(RecipesTableSchema.SCHEMA, reader); + } +} diff --git a/migrator/src/main/java/com/google/phonenumbers/migrator/MigrationJob.java b/migrator/src/main/java/com/google/phonenumbers/migrator/MigrationJob.java new file mode 100644 index 000000000..7b1800ae7 --- /dev/null +++ b/migrator/src/main/java/com/google/phonenumbers/migrator/MigrationJob.java @@ -0,0 +1,427 @@ +/* + * Copyright (C) 2020 The Libphonenumber Authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.phonenumbers.migrator; + +import com.google.common.collect.*; +import com.google.i18n.phonenumbers.metadata.DigitSequence; +import com.google.i18n.phonenumbers.metadata.RangeSpecification; +import com.google.i18n.phonenumbers.metadata.RangeTree; +import com.google.i18n.phonenumbers.metadata.model.RangesTableSchema; +import com.google.i18n.phonenumbers.metadata.table.Column; +import com.google.i18n.phonenumbers.metadata.table.CsvTable; +import com.google.i18n.phonenumbers.metadata.table.RangeKey; +import com.google.i18n.phonenumbers.metadata.table.RangeTable; +import java.io.FileWriter; +import java.io.IOException; +import java.io.OutputStream; +import java.util.Optional; +import com.google.common.base.Preconditions; +import java.util.stream.Stream; + +/** + * Represents a migration operation for a given country where each {@link MigrationJob} contains + * a list of {@link MigrationEntry}'s to be migrated as well as the {@link CsvTable} which will + * hold the available recipes that can be performed on the range. Each MigrationEntry has + * the E.164 {@link DigitSequence} representation of a number along with the raw input + * String originally entered. Only recipes from the given BCP-47 countryCode will be used. + */ +public final class MigrationJob { + + private final CsvTable rangesTable; + private final CsvTable recipesTable; + private final ImmutableList migrationEntries; + private final DigitSequence countryCode; + /** + * If true, when a {@link MigrationReport} is exported, the migrated version + * of numbers are written to file regardless of if the migrated number was seen as valid or invalid. + * By default, when a migration results in an invalid number for the given countryCode, the + * original number is written to file. + */ + private final boolean exportInvalidMigrations; + + MigrationJob(ImmutableList migrationEntries, + DigitSequence countryCode, + CsvTable recipesTable, + CsvTable rangesTable, + boolean exportInvalidMigrations) { + this.migrationEntries = migrationEntries; + this.countryCode = countryCode; + this.recipesTable = recipesTable; + this.rangesTable = rangesTable; + this.exportInvalidMigrations = exportInvalidMigrations; + } + + public DigitSequence getCountryCode() { + return countryCode; + } + + public CsvTable getRecipesCsvTable() { + return recipesTable; + } + + public RangeTable getRecipesRangeTable() { + return RecipesTableSchema.toRangeTable(recipesTable); + } + + public Stream getMigrationEntries() { + return migrationEntries.stream(); + } + + /** + * Retrieves all migratable numbers from the numberRange and attempts to migrate them with recipes + * from the recipesTable that belong to the given country code. + */ + public MigrationReport getMigrationReportForCountry() { + ImmutableList migratableRange = MigrationUtils + .getMigratableRangeByCountry(getRecipesRangeTable(), countryCode, getMigrationEntries()) + .collect(ImmutableList.toImmutableList()); + + ImmutableList.Builder migratedResults = ImmutableList.builder(); + for (MigrationEntry entry : migratableRange) { + MigrationUtils + .findMatchingRecipe(getRecipesCsvTable(), countryCode, entry.getSanitizedNumber()) + .ifPresent(recipe -> migratedResults.add(migrate(entry.getSanitizedNumber(), recipe, entry))); + } + Stream untouchedEntries = getMigrationEntries() + .filter(entry -> !migratableRange.contains(entry)); + + if (rangesTable == null) { + /* + * MigrationJob's with no rangesTable are based on a custom recipe file. This means there is no + * concept of invalid migrations so all migrations can just be seen as valid. + */ + return new MigrationReport(untouchedEntries, + ImmutableMap.of("Valid", migratedResults.build(), "Invalid", ImmutableList.of())); + } + return new MigrationReport(untouchedEntries, verifyMigratedNumbers(migratedResults.build())); + } + + /** + * Retrieves all migratable numbers from the numberRange that can be migrated using the given + * recipeKey and attempts to migrate them with recipes from the recipesTable that belong to the + * given country code. + */ + public Optional getMigrationReportForRecipe(RangeKey recipeKey) { + ImmutableMap, Object> recipeRow = getRecipesCsvTable().getRow(recipeKey); + ImmutableList migratableRange = MigrationUtils + .getMigratableRangeByRecipe(getRecipesCsvTable(), recipeKey, getMigrationEntries()) + .collect(ImmutableList.toImmutableList()); + ImmutableList.Builder migratedResults = ImmutableList.builder(); + + if (!recipeRow.get(RecipesTableSchema.COUNTRY_CODE).equals(countryCode)) { + return Optional.empty(); + } + migratableRange.forEach(entry -> migratedResults + .add(migrate(entry.getSanitizedNumber(), recipeRow, entry))); + + Stream untouchedEntries = getMigrationEntries() + .filter(entry -> !migratableRange.contains(entry)); + if (rangesTable == null) { + /* + * MigrationJob's with no rangesTable are based on a custom recipe file. This means there is no + * concept of invalid migrations so all migrations can just be seen as valid. + */ + return Optional.of(new MigrationReport(untouchedEntries, + ImmutableMap.of("Valid", migratedResults.build(), "Invalid", ImmutableList.of()))); + } + return Optional + .of(new MigrationReport(untouchedEntries, verifyMigratedNumbers(migratedResults.build()))); + } + + /** + * Takes a given number and migrates it using the given matching recipe row. If the given recipe + * is not a final migration, the method is recursively called with the recipe that matches the new + * migrated number until a recipe that produces a final migration (a recipe that results in the + * new format being valid and dialable) has been used. Once this occurs, the {@link MigrationResult} + * is returned. + * + * @throws IllegalArgumentException if the 'Old Format' value in the given recipe row does not + * match the number to migrate. This means that the 'Old Format' value cannot be represented by + * the given recipes 'Old Prefix' and 'Old Length'. + * @throws RuntimeException when the given recipe is not a final migration and a recipe cannot be + * found in the recipesTable to match the resulting number from the initial migrating recipe. + */ + private MigrationResult migrate(DigitSequence migratingNumber, + ImmutableMap, Object> recipeRow, + MigrationEntry migrationEntry) { + String oldFormat = (String) recipeRow.get(RecipesTableSchema.OLD_FORMAT); + String newFormat = (String) recipeRow.get(RecipesTableSchema.NEW_FORMAT); + + Preconditions.checkArgument(RangeSpecification.parse(oldFormat).matches(migratingNumber), + "value '%s' in column 'Old Format' cannot be represented by its given" + + " recipe key (Old Prefix + Old Length)", oldFormat); + + DigitSequence migratedVal = getMigratedValue(migratingNumber.toString(), oldFormat, newFormat); + + /* + * Only recursively migrate when the recipe explicitly states it is not a final migration. + * Custom recipes have no concept of an Is_Final_Migration column so their value will be seen + * as null here. In such cases, the tool should not look for another recipe after a migration. + */ + if (Boolean.FALSE.equals(recipeRow.get(RecipesTableSchema.IS_FINAL_MIGRATION))) { + ImmutableMap, Object> nextRecipeRow = + MigrationUtils.findMatchingRecipe(getRecipesCsvTable(), countryCode, migratedVal) + .orElseThrow(() -> new RuntimeException( + "A multiple migration was required for the stale number '" + migrationEntry + .getOriginalNumber() + "' but no other recipe could be found after migrating " + + "the number into +" + migratedVal)); + + return migrate(migratedVal, nextRecipeRow, migrationEntry); + } + return MigrationResult.create(migratedVal, migrationEntry); + } + + /** + * Converts a stale number into the new migrated format based on the information from the given + * oldFormat and newFormat values. + */ + private DigitSequence getMigratedValue(String staleString, String oldFormat, String newFormat) { + StringBuilder migratedValue = new StringBuilder(); + int newFormatPointer = 0; + + for (int i = 0; i < oldFormat.length(); i++) { + if (!Character.isDigit(oldFormat.charAt(i))) { + migratedValue.append(staleString.charAt(i)); + } + } + for (int i = 0; i < Math.max(oldFormat.length(), newFormat.length()); i++) { + if (i < newFormat.length() && i == newFormatPointer + && Character.isDigit(newFormat.charAt(i))) { + do { + migratedValue.insert(newFormatPointer, newFormat.charAt(newFormatPointer++)); + } while (newFormatPointer < newFormat.length() + && Character.isDigit(newFormat.charAt(newFormatPointer))); + } + if (newFormatPointer == i) { + newFormatPointer++; + } + } + return DigitSequence.of(migratedValue.toString()); + } + + /** + * Given a list of {@link MigrationResult}'s, returns a map detailing which migrations resulted in + * valid phone numbers based on the given rangesTable data. The map will contain to entries; an + * entry with the key 'Valid' with a list of the valid migrations and an entry with the key + * 'Invalid', with a list of the invalid migrations from the overall list. + */ + private ImmutableMap> verifyMigratedNumbers( + ImmutableList migrations) { + ImmutableList.Builder validMigrations = ImmutableList.builder(); + ImmutableList.Builder invalidMigrations = ImmutableList.builder(); + + RangeTree validRanges = RangesTableSchema.toRangeTable(rangesTable).getAllRanges(); + for (MigrationResult migration : migrations) { + DigitSequence migratedNum = migration.getMigratedNumber(); + if (migratedNum.length() <= countryCode.length()) { + invalidMigrations.add(migration); + continue; + } + if(validRanges.contains(migratedNum.last(migratedNum.length() - countryCode.length()))) { + validMigrations.add(migration); + } else { + invalidMigrations.add(migration); + } + } + return ImmutableMap.of("Valid", validMigrations.build(), "Invalid", invalidMigrations.build()); + } + + /** + * Represents the results of a migration when calling {@link #getMigrationReportForCountry()} + * or {@link #getMigrationReportForRecipe(RangeKey)} + */ + public final class MigrationReport { + private final ImmutableList untouchedEntries; + private final ImmutableList validMigrations; + private final ImmutableList invalidMigrations; + + private MigrationReport(Stream untouchedEntries, + ImmutableMap> migratedEntries) { + this.untouchedEntries = untouchedEntries.collect(ImmutableList.toImmutableList()); + this.validMigrations = migratedEntries.get("Valid"); + this.invalidMigrations = migratedEntries.get("Invalid"); + } + + public DigitSequence getCountryCode() { + return countryCode; + } + + /** + * Returns the Migration results which were seen as valid when queried against the rangesTable + * containing valid number representations for the given countryCode. + * + * Note: for customRecipe migrations, there is no concept of invalid migrations so all + * {@link MigrationEntry}'s that were migrated will be seen as valid. + */ + public ImmutableList getValidMigrations() { + return validMigrations; + } + + /** + * Returns the Migration results which were seen as invalid when queried against the given + * rangesTable. + * + * Note: for customRecipe migrations, there is no concept of invalid migrations so all + * {@link MigrationEntry}'s that were migrated will be seen as valid. + */ + public ImmutableList getInvalidMigrations() { + return invalidMigrations; + } + + /** + * Returns the Migration entry's that were not migrated but were seen as being already valid + * when querying against the rangesTable. Custom recipe migrations do not have range tables so + * this list will be empty when called from such instance. + */ + public ImmutableList getUntouchedEntries() { + return untouchedEntries; + } + + /** + * Creates a text file of the new number list after a migration has been performed. + * + * @param fileName: the given suffix of the new file to be created. + */ + public String exportToFile(String fileName) throws IOException { + String newFileLocation = "+" + countryCode + "_Migration_" + fileName; + FileWriter fw = new FileWriter(newFileLocation); + fw.write(toString()); + fw.close(); + return newFileLocation; + } + + /** + * Returns the content for the given migration. Numbers that were not migrated are added in their original format as + * well migrated numbers that were seen as being invalid, unless the migration job is set to exportInvalidMigrations. + * Successfully migrated numbers will be added in their new format. + */ + public String toString() { + StringBuilder fileContent = new StringBuilder(); + for (MigrationResult result : validMigrations) { + fileContent.append("+").append(result.getMigratedNumber()).append("\n"); + } + for (MigrationEntry entry : untouchedEntries) { + fileContent.append(entry.getOriginalNumber()).append("\n"); + } + if (exportInvalidMigrations && invalidMigrations.size() > 0) { + fileContent.append("\nInvalid migrations due to an issue in either the used internal recipe or the internal +") + .append(countryCode).append(" valid metadata range:\n"); + } + for (MigrationResult result : invalidMigrations) { + String number = exportInvalidMigrations ? "+" + result.getMigratedNumber() : + result.getMigrationEntry().getOriginalNumber(); + fileContent.append(number).append("\n"); + } + return fileContent.toString(); + } + + /** + * Queries the list of numbers that were not migrated and returns numbers from the list which are + * seen as valid based on the given rangesTable for the given countryCode. + */ + public ImmutableList getValidUntouchedEntries() { + if (rangesTable == null) { + return ImmutableList.of(); + } + ImmutableList.Builder validEntries = ImmutableList.builder(); + RangeTree validRanges = RangesTableSchema.toRangeTable(rangesTable).getAllRanges(); + + for (MigrationEntry entry : untouchedEntries) { + DigitSequence sanitizedNum = entry.getSanitizedNumber(); + if (sanitizedNum.length() <= countryCode.length() || + !sanitizedNum.first(countryCode.length()).equals(countryCode)) { + continue; + } + if(validRanges.contains(sanitizedNum.last(sanitizedNum.length() - countryCode.length()))) { + validEntries.add(entry); + } + } + return validEntries.build(); + } + + /** + * Maps all migrated numbers, whether invalid or valid, to the recipe from the recipesTable that + * was used to migrate them. + */ + public Multimap, Object>, MigrationResult> getAllRecipesUsed() { + Multimap, Object>, MigrationResult> recipeToNumbers = ArrayListMultimap + .create(); + + for (MigrationResult migration : validMigrations) { + MigrationUtils.findMatchingRecipe(recipesTable, countryCode, + migration.getMigrationEntry().getSanitizedNumber()) + .ifPresent(recipe -> recipeToNumbers.put(recipe, migration)); + } + for (MigrationResult migration : invalidMigrations) { + MigrationUtils.findMatchingRecipe(recipesTable, countryCode, + migration.getMigrationEntry().getSanitizedNumber()) + .ifPresent(recipe -> recipeToNumbers.put(recipe, migration)); + } + return recipeToNumbers; + } + + /** Prints to console the details of the given migration. */ + public void printMetrics() { + int migratedCount = validMigrations.size() + invalidMigrations.size(); + int totalCount = untouchedEntries.size() + migratedCount; + + System.out.println("\nMetrics:"); + System.out.println("* " + migratedCount + " out of the " + totalCount + " inputted numbers " + + "were/was migrated"); + + if (rangesTable == null) { + /* + * MigrationJob's with no rangesTable are based on a custom recipe file. This means there + * is no concept of invalid/valid migrations so all migrations can just be listed. + */ + System.out.println("* Migrated numbers:"); + validMigrations.forEach(val -> System.out.println("\t" + val)); + System.out.println("\n* Untouched number(s):"); + untouchedEntries.forEach(val -> System.out.println("\t" + val.getOriginalNumber())); + + } else { + ImmutableList validUntouchedEntries = getValidUntouchedEntries(); + System.out.println("* " + validMigrations.size() + " out of the " + migratedCount + + " migrated numbers were/was verified as being in a valid, dialable format based on our " + + "data for the given country"); + System.out.println("* " + validUntouchedEntries.size() + " out of the " + + untouchedEntries.size() + " non-migrated numbers were/was already in a valid, dialable " + + "format based on our data for the given country"); + + System.out.println("\n* Valid number(s):"); + validMigrations.forEach(val -> System.out.println("\t" + val)); + validUntouchedEntries.forEach(val -> System.out.println("\t" + val.getOriginalNumber() + + " (untouched)")); + + System.out.println("\n* Invalid migrated number(s):"); + invalidMigrations.forEach(val -> System.out.println("\t" + val)); + + System.out.println("\n* Untouched number(s):"); + /* converted into a Set to allow for constant time contains() method. Can only be converted into a set once all its + numbers have been printed out above because duplicate numbers could have been entered for migration and users + should still receive all duplicates. */ + ImmutableSet validUntouchedEntriesSet = ImmutableSet.copyOf(validUntouchedEntries); + untouchedEntries.forEach(val -> { + if (validUntouchedEntriesSet.contains(val)) { + System.out.println("\t" + val.getOriginalNumber() + " (already valid)"); + } else { + System.out.println("\t" + val.getOriginalNumber()); + } + }); + } + } + } +} diff --git a/migrator/src/main/java/com/google/phonenumbers/migrator/MigrationResult.java b/migrator/src/main/java/com/google/phonenumbers/migrator/MigrationResult.java new file mode 100644 index 000000000..c8941a046 --- /dev/null +++ b/migrator/src/main/java/com/google/phonenumbers/migrator/MigrationResult.java @@ -0,0 +1,39 @@ +/* + * Copyright (C) 2020 The Libphonenumber Authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.phonenumbers.migrator; + +import com.google.auto.value.AutoValue; +import com.google.i18n.phonenumbers.metadata.DigitSequence; + +/** + * Representation of the result for a given MigrationEntry. Contains the {@link MigrationEntry} with + * its migrated E.164 {@link DigitSequence} value. + */ +@AutoValue +public abstract class MigrationResult { + public abstract DigitSequence getMigratedNumber(); + public abstract MigrationEntry getMigrationEntry(); + + public static MigrationResult create(DigitSequence migratedNumber, + MigrationEntry migrationEntry) { + return new AutoValue_MigrationResult(migratedNumber, migrationEntry); + } + + @Override + public String toString() { + return getMigrationEntry().getOriginalNumber() + " -> +" + getMigratedNumber(); + } +} \ No newline at end of file diff --git a/migrator/src/main/java/com/google/phonenumbers/migrator/MigrationUtils.java b/migrator/src/main/java/com/google/phonenumbers/migrator/MigrationUtils.java new file mode 100644 index 000000000..e3d05341f --- /dev/null +++ b/migrator/src/main/java/com/google/phonenumbers/migrator/MigrationUtils.java @@ -0,0 +1,88 @@ +/* + * Copyright (C) 2020 The Libphonenumber Authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.phonenumbers.migrator; + +import com.google.common.collect.ImmutableMap; +import com.google.i18n.phonenumbers.metadata.DigitSequence; +import com.google.i18n.phonenumbers.metadata.RangeTree; +import com.google.i18n.phonenumbers.metadata.table.Column; +import com.google.i18n.phonenumbers.metadata.table.CsvTable; +import com.google.i18n.phonenumbers.metadata.table.RangeKey; +import com.google.i18n.phonenumbers.metadata.table.RangeTable; +import java.util.Optional; +import java.util.stream.Stream; + +/** Utilities for migration tool. */ +public final class MigrationUtils { + + /** + * Returns the entries within migrationEntries that can be migrated using the given recipe. This + * method will not perform migrations and as a result, the validity of migrations using the given + * recipe cannot be verified. + * + * @param recipeKey: the key of the recipe that is being checked + * @throws IllegalArgumentException if there is no row in the recipesTable with the given + * recipeKey + */ + public static Stream getMigratableRangeByRecipe(CsvTable recipesTable, + RangeKey recipeKey, + Stream migrationEntries) { + if (!recipesTable.containsRow(recipeKey)) { + throw new IllegalArgumentException( + recipeKey + " does not match any recipe row in the given recipes table"); + } + + return migrationEntries + .filter(entry -> recipeKey.asRangeTree().contains(entry.getSanitizedNumber())); + } + + /** + * Returns the sub range of entries within migrationEntries that can be migrated using any recipe + * from the {@link CsvTable} recipesTable that matches the specified BCP-47 country code. This + * method will not perform migrations and as a result, the validity of migrations using the given + * recipesTable cannot be verified. + */ + public static Stream getMigratableRangeByCountry(RangeTable recipesTable, + DigitSequence countryCode, + Stream migrationEntries) { + + RangeTree countryRecipes = recipesTable + .getRanges(RecipesTableSchema.COUNTRY_CODE, countryCode); + + return migrationEntries + .filter(entry -> countryRecipes.contains(entry.getSanitizedNumber())); + } + + /** + * Returns the {@link CsvTable} row for the given recipe in a recipes table that can be used to + * migrate the given {@link DigitSequence}. The found recipe must also be linked to the given + * country code to ensure that recipes from incorrect countries are not used to migrated a given + * number. + */ + public static Optional, Object>> findMatchingRecipe( + CsvTable recipesTable, + DigitSequence countryCode, + DigitSequence number) { + + for (RangeKey recipeKey : recipesTable.getKeys()) { + if (recipeKey.contains(number, number.length()) && recipesTable.getRow(recipeKey) + .get(RecipesTableSchema.COUNTRY_CODE).equals(countryCode)) { + return Optional.of(recipesTable.getRow(recipeKey)); + } + } + return Optional.empty(); + } +} diff --git a/migrator/src/main/java/com/google/phonenumbers/migrator/RecipesTableSchema.java b/migrator/src/main/java/com/google/phonenumbers/migrator/RecipesTableSchema.java new file mode 100644 index 000000000..6d824299e --- /dev/null +++ b/migrator/src/main/java/com/google/phonenumbers/migrator/RecipesTableSchema.java @@ -0,0 +1,168 @@ +/* + * Copyright (C) 2020 The Libphonenumber Authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.phonenumbers.migrator; + +import com.google.common.collect.ImmutableMap; +import com.google.i18n.phonenumbers.metadata.DigitSequence; +import com.google.i18n.phonenumbers.metadata.RangeSpecification; +import com.google.i18n.phonenumbers.metadata.model.RangesTableSchema; +import com.google.i18n.phonenumbers.metadata.table.Change; +import com.google.i18n.phonenumbers.metadata.table.Column; +import com.google.i18n.phonenumbers.metadata.table.CsvKeyMarshaller; +import com.google.i18n.phonenumbers.metadata.table.CsvSchema; +import com.google.i18n.phonenumbers.metadata.table.CsvTable; +import com.google.i18n.phonenumbers.metadata.table.RangeKey; +import com.google.i18n.phonenumbers.metadata.table.RangeTable; +import com.google.i18n.phonenumbers.metadata.table.RangeTable.OverwriteMode; +import com.google.i18n.phonenumbers.metadata.table.Schema; +import java.util.Collections; +import java.util.List; +import java.util.Optional; +import java.util.Set; + +/** + * The schema of the standard "Recipes" table with rows keyed by {@link RangeKey} and columns: + *
    + *
  1. {@link #OLD_FORMAT}: The original format of the represented range in the row to be changed. + * 'x' characters represent indexes that do not need to be changed in a number within the range + * and actual digits in the string are values that need to be removed or replaced. (e.g. xx98xx). + * The length of this string must match the lengths of (DigitSequence)'s produced by the Row Key. + *
  2. {@link #NEW_FORMAT}: The migrated format of the represented range in the row. 'x' characters + * represent indexes that do not need to be changed in a number within the range and actual + * digits in the string are values that need to be added at that given index. + *
  3. {@link #IS_FINAL_MIGRATION}: A boolean indicating whether the given recipe row would result + * in the represented range being migrated into up to date, dialable formats. Recipes which + * do not will require the newly formatted range to be migrated again using another matching + * recipe. + *
  4. {@link #COUNTRY_CODE}: The BCP-47 country code in which a given recipe corresponds to. + *
  5. {@link #DESCRIPTION}: TThe explanation of a migration recipe in words. + *
+ * + *

Rows keys are serialized via the marshaller and produce leading columns: + *

    + *
  1. {@code "Old Prefix"}: The prefix (RangeSpecification) for the original ranges in a row + * (e.g. "44123"). + *
  2. {@code "Old Length"}: The length for the original ranges in a row (e.g. "9", "8" or "5"). + *
+ */ +public class RecipesTableSchema { + + /** The format of the original numbers in a given range. */ + public static final Column OLD_FORMAT = Column.ofString("Old Format"); + + /** The new format of the migrated numbers in a given range. */ + public static final Column NEW_FORMAT = Column.ofString("New Format"); + + /** The BCP-47 country code the given recipe belongs to. */ + public static final Column COUNTRY_CODE = + Column.create(DigitSequence.class, "Country Code", DigitSequence.empty(), DigitSequence::of); + + /** Indicates whether a given recipe will result in a valid, dialable range */ + public static final Column IS_FINAL_MIGRATION = Column.ofBoolean("Is Final Migration"); + + /** The explanation of a migration recipe in words. */ + public static final Column DESCRIPTION = Column.ofString("Description"); + + /** Marshaller for constructing CsvTable from RangeTable. */ + private static final CsvKeyMarshaller MARSHALLER = new CsvKeyMarshaller<>( + RangesTableSchema::write, + // uses a read method that will only allow a single numerical value in the 'Old Length' column + RecipesTableSchema::read, + Optional.of(RangeKey.ORDERING), + "Old Prefix", + "Old Length"); + + /** + * Instantiates a {@link RangeKey} from the prefix and length columns of a given recipe + * row. + * + * @throws IllegalArgumentException when the 'Old Length' value is anything other than a number + */ + public static RangeKey read(List parts) { + Set rangeKeyLength; + + try { + rangeKeyLength = Collections.singleton(Integer.parseInt(parts.get(1))); + } catch (NumberFormatException e) { + throw new IllegalArgumentException("Invalid number '" + parts.get(1) + "' in column 'Old Length'"); + } + return RangeKey.create(RangeSpecification.parse(parts.get(0)), rangeKeyLength); + } + + /** The columns for the serialized CSV table. */ + private static final Schema CSV_COLUMNS = + Schema.builder() + .add(COUNTRY_CODE) + .add(OLD_FORMAT) + .add(NEW_FORMAT) + .add(IS_FINAL_MIGRATION) + .add(DESCRIPTION) + .build(); + + /** Schema instance defining the ranges CSV table. */ + public static final CsvSchema SCHEMA = CsvSchema.of(MARSHALLER, CSV_COLUMNS); + + /** The non-key columns of a range table. */ + private static final Schema RANGE_COLUMNS = + Schema.builder() + .add(COUNTRY_CODE) + .add(OLD_FORMAT) + .add(NEW_FORMAT) + .add(IS_FINAL_MIGRATION) + .add(DESCRIPTION) + .build(); + + /** + * Converts a {@link RangeKey} based {@link CsvTable} to a {@link RangeTable}, preserving the + * original table columns. The {@link CsvSchema} of the returned table is not guaranteed to be + * the {@link #SCHEMA} instance if the given table had different columns. + */ + public static RangeTable toRangeTable(CsvTable csv) { + RangeTable.Builder out = RangeTable.builder(RANGE_COLUMNS); + for (RangeKey k : csv.getKeys()) { + Change.Builder change = Change.builder(k.asRangeTree()); + csv.getRow(k).forEach(change::assign); + out.apply(change.build(), OverwriteMode.NEVER); + } + return out.build(); + } + + /** + * Converts a {@link RangeTable} to a {@link CsvTable}, using {@link RangeKey}s as row keys and + * preserving the original table columns. The {@link CsvSchema} of the returned table is not + * guaranteed to be the {@link #SCHEMA} instance if the given table had different columns. + */ + @SuppressWarnings("unchecked") + public static CsvTable toCsv(RangeTable table) { + CsvTable.Builder csv = CsvTable.builder(SCHEMA); + for (Change c : table.toChanges()) { + for (RangeKey k : RangeKey.decompose(c.getRanges())) { + c.getAssignments().forEach(a -> csv.put(k, a)); + } + } + return csv.build(); + } + + /** Converts recipe into format more human-friendly than the default ImmutableMap toString(). */ + public static String formatRecipe(ImmutableMap, Object> recipe) { + StringBuilder formattedRecipe = new StringBuilder(); + for (Column column : recipe.keySet()) { + String columnValue = column.getName() + ": " + recipe.get(column) + " | "; + formattedRecipe.append(columnValue); + } + return formattedRecipe.toString(); + } +} diff --git a/migrator/src/test/java/com/google/phonenumbers/migrator/CommandLineMainTest.java b/migrator/src/test/java/com/google/phonenumbers/migrator/CommandLineMainTest.java new file mode 100644 index 000000000..92aa7b4eb --- /dev/null +++ b/migrator/src/test/java/com/google/phonenumbers/migrator/CommandLineMainTest.java @@ -0,0 +1,89 @@ +/* + * Copyright (C) 2020 The Libphonenumber Authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.phonenumbers.migrator; + +import static com.google.common.truth.Truth.assertThat; + +import org.junit.Assert; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.JUnit4; +import picocli.CommandLine; +import picocli.CommandLine.MissingParameterException; +import picocli.CommandLine.MutuallyExclusiveArgsException; + +@RunWith(JUnit4.class) +public class CommandLineMainTest { + private static final String TEST_COUNTRY_CODE = "44"; + private static final String TEST_NUMBER_INPUT = "12345"; + private static final String TEST_FILE_INPUT = "../test-file-path.txt"; + + @Test + public void createMigrationJob_noNumberInputSpecified_expectException() { + String[] args = ("--countryCode=" + TEST_COUNTRY_CODE).split(" "); + try { + CommandLine.populateCommand(new CommandLineMain(), args); + Assert.fail("Expected RuntimeException and did not receive"); + } catch (MissingParameterException e) { + assertThat(e).isInstanceOf(MissingParameterException.class); + } + } + + @Test + public void createMigrationJob_numberAndFile_expectException() { + String[] args = ("--countryCode=" + TEST_COUNTRY_CODE + " --number=" + TEST_NUMBER_INPUT + + " --file=" + TEST_FILE_INPUT).split(" "); + try { + CommandLine.populateCommand(new CommandLineMain(), args); + Assert.fail("Expected RuntimeException and did not receive"); + } catch (MutuallyExclusiveArgsException e) { + assertThat(e).isInstanceOf(MutuallyExclusiveArgsException.class); + } + } + + @Test + public void createFromNumberString_expectSufficientArguments() { + String[] args = ("--countryCode=" + TEST_COUNTRY_CODE + " --number=" + TEST_NUMBER_INPUT) + .split(" "); + CommandLineMain p = CommandLine.populateCommand(new CommandLineMain(), args); + assertThat(p.countryCode).matches(TEST_COUNTRY_CODE); + assertThat(p.numberInput.number).matches(TEST_NUMBER_INPUT); + assertThat(p.numberInput.file).isNull(); + } + + @Test + public void createFromPath_expectSufficientArguments() { + String[] args = ("--countryCode="+ TEST_COUNTRY_CODE +" --file="+TEST_FILE_INPUT) + .split(" "); + CommandLineMain p = CommandLine.populateCommand(new CommandLineMain(), args); + assertThat(p.countryCode).matches(TEST_COUNTRY_CODE); + assertThat(p.numberInput.file).matches(TEST_FILE_INPUT); + assertThat(p.numberInput.number).isNull(); + } + + @Test + public void createMigrationJob_exportInvalidMigrationsAndCustomRecipe_expectException() { + String[] args = ("--countryCode=" + TEST_COUNTRY_CODE + " --number=" + TEST_NUMBER_INPUT + + " --exportInvalidMigrations --customRecipe=" + TEST_FILE_INPUT).split(" "); + try { + CommandLine.populateCommand(new CommandLineMain(), args); + Assert.fail("Expected MutuallyExclusiveArgsException and did not receive"); + } catch (MutuallyExclusiveArgsException e) { + assertThat(e.getMessage()).contains("mutually exclusive"); + assertThat(e.getMessage()).contains("--exportInvalidMigrations, --customRecipe"); + } + } +} diff --git a/migrator/src/test/java/com/google/phonenumbers/migrator/MetadataZipFileReaderTest.java b/migrator/src/test/java/com/google/phonenumbers/migrator/MetadataZipFileReaderTest.java new file mode 100644 index 000000000..5852a2b59 --- /dev/null +++ b/migrator/src/test/java/com/google/phonenumbers/migrator/MetadataZipFileReaderTest.java @@ -0,0 +1,66 @@ +/* + * Copyright (C) 2020 The Libphonenumber Authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.phonenumbers.migrator; + +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.truth.Truth8.assertThat; + +import com.google.i18n.phonenumbers.metadata.DigitSequence; +import com.google.i18n.phonenumbers.metadata.table.CsvTable; +import com.google.i18n.phonenumbers.metadata.table.RangeKey; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.NoSuchFileException; +import java.nio.file.Paths; +import java.util.Optional; +import org.junit.Assert; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.JUnit4; + +@RunWith(JUnit4.class) +public class MetadataZipFileReaderTest { + + private static final String TEST_DATA_PATH = "./src/test/java/com/google/phonenumbers/migrator/testing/testData/"; + + @Test + public void createInstance_invalidFileLocation_expectException() { + String fileLocation = "invalid-zipfile-location"; + try { + MetadataZipFileReader.of(Files.newInputStream(Paths.get(fileLocation))); + Assert.fail("Expected IOException and did not receive"); + } catch (IOException e) { + assertThat(e).isInstanceOf(NoSuchFileException.class); + assertThat(e).hasMessageThat().contains(fileLocation); + } + } + + @Test + public void importTable_countryCodeInZip_expectCsvTable() throws IOException { + String fileLocation = TEST_DATA_PATH + "testMetadataZip.zip"; + MetadataZipFileReader validZip = MetadataZipFileReader.of(Files.newInputStream(Paths.get(fileLocation))); + Optional> regionTable = validZip.importCsvTable(DigitSequence.of("1")); + assertThat(regionTable).isPresent(); + } + + @Test + public void importTable_countryCodeNotInZip_expectEmptyCsvTable() throws IOException { + String fileLocation = TEST_DATA_PATH + "testMetadataZip.zip/"; + MetadataZipFileReader validZip = MetadataZipFileReader.of(Files.newInputStream(Paths.get(fileLocation))); + Optional> regionTable = validZip.importCsvTable(DigitSequence.of("2")); + assertThat(regionTable).isEmpty(); + } +} diff --git a/migrator/src/test/java/com/google/phonenumbers/migrator/MigrationFactoryTest.java b/migrator/src/test/java/com/google/phonenumbers/migrator/MigrationFactoryTest.java new file mode 100644 index 000000000..466a51f69 --- /dev/null +++ b/migrator/src/test/java/com/google/phonenumbers/migrator/MigrationFactoryTest.java @@ -0,0 +1,104 @@ +/* + * Copyright (C) 2020 The Libphonenumber Authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.phonenumbers.migrator; + +import static com.google.common.truth.Truth.assertThat; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.NoSuchFileException; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.stream.Collectors; +import org.junit.Assert; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.JUnit4; + +@RunWith(JUnit4.class) +public class MigrationFactoryTest { + + private static final String TEST_DATA_PATH = "./src/test/java/com/google/phonenumbers/migrator/testing/testData/"; + private static final Path RECIPES_PATH = Paths.get(TEST_DATA_PATH + "testRecipesFile.csv"); + + @Test + public void createFromFilePath_invalidPathLocation_expectException() { + String fileLocation = "invalid-path-location"; + try { + MigrationFactory.createMigration(Paths.get(fileLocation), "44", false); + Assert.fail("Expected IOException and did not receive"); + } catch (IOException e) { + assertThat(e).isInstanceOf(NoSuchFileException.class); + assertThat(e).hasMessageThat().contains(fileLocation); + } + } + + @Test + public void createFromFilePath_validPathLocation_expectValidFields() throws IOException { + Path fileLocation = Paths.get(TEST_DATA_PATH + "testNumbersFile.txt"); + String countryCode = "44"; + MigrationJob mj = MigrationFactory.createCustomRecipeMigration(fileLocation, countryCode, MigrationFactory + .importRecipes(Files.newInputStream(RECIPES_PATH))); + + assertThat(mj.getMigrationEntries().map(MigrationEntry::getOriginalNumber) + .collect(Collectors.toList())) + .containsExactlyElementsIn(Files.readAllLines(fileLocation)); + assertThat(mj.getCountryCode().toString()).matches(countryCode); + } + + @Test + public void createFromNumberString_invalidNumberFormat_expectException() { + String numberInput = "+44 one2 34 56"; + String sanitizedNumber = "44one23456"; + try { + MigrationFactory.createMigration(numberInput, "44"); + Assert.fail("Expected RuntimeException and did not receive"); + } catch (IllegalArgumentException e) { + assertThat(e).isInstanceOf(IllegalArgumentException.class); + assertThat(e).hasMessageThat().contains(sanitizedNumber); + } catch (IOException e) { + // IOException is not being tested here + e.printStackTrace(); + Assert.fail("Expected RuntimeException and did not receive"); + } + } + + @Test + public void createFromNumberString_validNumberFormat_expectValidFields() throws IOException { + String numberString = "12345"; + String countryCode = "1"; + MigrationJob mj = MigrationFactory.createCustomRecipeMigration(numberString, countryCode, MigrationFactory + .importRecipes(Files.newInputStream(RECIPES_PATH))); + + assertThat(mj.getMigrationEntries().map(MigrationEntry::getOriginalNumber) + .collect(Collectors.toList())) + .containsExactly(numberString); + assertThat(mj.getCountryCode().toString()).matches(countryCode); + } + + @Test + public void createWithCustomRecipes_invalidPathLocation_expectException() { + Path fileLocation = Paths.get("invalid-recipe-location"); + try { + MigrationFactory.createCustomRecipeMigration("12345", "44", MigrationFactory + .importRecipes(Files.newInputStream(fileLocation))); + Assert.fail("Expected IOException and did not receive"); + } catch (IOException e) { + assertThat(e).isInstanceOf(NoSuchFileException.class); + assertThat(e).hasMessageThat().contains(fileLocation.toString()); + } + } +} diff --git a/migrator/src/test/java/com/google/phonenumbers/migrator/MigrationJobTest.java b/migrator/src/test/java/com/google/phonenumbers/migrator/MigrationJobTest.java new file mode 100644 index 000000000..898d5c7f4 --- /dev/null +++ b/migrator/src/test/java/com/google/phonenumbers/migrator/MigrationJobTest.java @@ -0,0 +1,313 @@ +/* + * Copyright (C) 2020 The Libphonenumber Authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.phonenumbers.migrator; + +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import com.google.i18n.phonenumbers.metadata.DigitSequence; +import com.google.i18n.phonenumbers.metadata.RangeSpecification; +import com.google.i18n.phonenumbers.metadata.RangeTree; +import com.google.i18n.phonenumbers.metadata.table.CsvTable; +import com.google.i18n.phonenumbers.metadata.table.RangeKey; +import com.google.phonenumbers.migrator.MigrationJob.MigrationReport; +import org.junit.Assert; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.JUnit4; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.Collections; +import java.util.List; +import java.util.Optional; +import java.util.stream.Collectors; + +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.truth.Truth8.assertThat; + +@RunWith(JUnit4.class) +public class MigrationJobTest { + + private static final String COUNTRY_CODE = "44"; + private static final String TEST_DATA_PATH = "./src/test/java/com/google/phonenumbers/migrator/testing/testData/"; + private static final Path RECIPES_PATH = Paths.get(TEST_DATA_PATH + "testRecipesFile.csv"); + + @Test + public void customRecipesMigration_expectMigrations() throws IOException { + String numbersPath = TEST_DATA_PATH + "testNumbersFile.txt"; + MigrationJob job = MigrationFactory + .createCustomRecipeMigration(Paths.get(numbersPath), COUNTRY_CODE, MigrationFactory + .importRecipes(Files.newInputStream(RECIPES_PATH))); + + MigrationReport report = job.getMigrationReportForCountry(); + assertThat(report.getValidMigrations()).isNotEmpty(); + } + + @Test + public void customRecipesMigration_noRecipesFromCountry_expectNoMigrations() throws IOException { + String numbersPath = TEST_DATA_PATH + "testNumbersFile.txt"; + String unsupportedCountry = "1"; + MigrationJob job = MigrationFactory + .createCustomRecipeMigration(Paths.get(numbersPath), unsupportedCountry, MigrationFactory + .importRecipes(Files.newInputStream(RECIPES_PATH))); + + MigrationReport report = job.getMigrationReportForCountry(); + assertThat(report.getValidMigrations()).isEmpty(); + } + + @Test + public void customRecipes_singleMigration_unsupportedRecipeKey_expectException() throws IOException { + String numbersPath = TEST_DATA_PATH + "testNumbersFile.txt"; + MigrationJob job = MigrationFactory + .createCustomRecipeMigration(Paths.get(numbersPath), COUNTRY_CODE, MigrationFactory + .importRecipes(Files.newInputStream(RECIPES_PATH))); + + RangeSpecification testRecipePrefix = RangeSpecification.from(DigitSequence.of("123")); + int testRecipeLength = 3; + RangeKey invalidKey = RangeKey.create(testRecipePrefix, Collections.singleton(testRecipeLength)); + + try { + job.getMigrationReportForRecipe(invalidKey); + Assert.fail("Expected RuntimeException and did not receive"); + } catch (IllegalArgumentException e) { + assertThat(e).isInstanceOf(IllegalArgumentException.class); + assertThat(e).hasMessageThat().contains(invalidKey.toString()); + } + } + + @Test + public void customRecipes_singleMigration_validKey_expectMigration() throws IOException { + String numbersPath = TEST_DATA_PATH + "testNumbersFile.txt"; + MigrationJob job = MigrationFactory + .createCustomRecipeMigration(Paths.get(numbersPath), COUNTRY_CODE, MigrationFactory + .importRecipes(Files.newInputStream(RECIPES_PATH))); + + RangeSpecification testRecipePrefix = RangeSpecification.from(DigitSequence.of("12")); + int testRecipeLength = 5; + RangeKey validKey = RangeKey.create(testRecipePrefix, Collections.singleton(testRecipeLength)); + + Optional migratedNums = job.getMigrationReportForRecipe(validKey); + assertThat(migratedNums).isPresent(); + } + + @Test + public void customRecipes_invalidOldFormatValue_expectException() throws IOException { + MigrationJob job = MigrationFactory + .createCustomRecipeMigration("13321", COUNTRY_CODE, MigrationFactory + .importRecipes(Files.newInputStream(RECIPES_PATH))); + + RangeSpecification testRecipePrefix = RangeSpecification.from(DigitSequence.of("13")); + int testRecipeLength = 5; + RangeKey recipeKey = RangeKey.create(testRecipePrefix, Collections.singleton(testRecipeLength)); + + try { + job.getMigrationReportForRecipe(recipeKey); + Assert.fail("Expected RuntimeException and did not receive"); + } catch (RuntimeException e) { + assertThat(e).isInstanceOf(IllegalArgumentException.class); + assertThat(e).hasMessageThat().contains("Old Format"); + } + } + + @Test + public void customRecipe_multipleMigration_nextRecipeNotFound_expectException() throws IOException { + String staleNumber = "10321"; + MigrationJob job = MigrationFactory + .createCustomRecipeMigration(staleNumber, COUNTRY_CODE, MigrationFactory + .importRecipes(Files.newInputStream(RECIPES_PATH))); + + RangeSpecification testRecipePrefix = RangeSpecification.from(DigitSequence.of("10")); + int testRecipeLength = 5; + RangeKey recipeKey = RangeKey.create(testRecipePrefix, Collections.singleton(testRecipeLength)); + + try { + job.getMigrationReportForRecipe(recipeKey); + Assert.fail("Expected RuntimeException and did not receive"); + } catch (RuntimeException e) { + assertThat(e).isInstanceOf(RuntimeException.class); + assertThat(e).hasMessageThat().contains("multiple migration"); + assertThat(e).hasMessageThat().contains(staleNumber); + } + } + + @Test + public void customRecipe_multipleMigration_expectMigration() throws IOException { + String staleNumber = "15321"; + String migratedNumber = "130211"; + MigrationJob job = MigrationFactory + .createCustomRecipeMigration(staleNumber, COUNTRY_CODE, MigrationFactory + .importRecipes(Files.newInputStream(RECIPES_PATH))); + + RangeSpecification testRecipePrefix = RangeSpecification.from(DigitSequence.of("15")); + int testRecipeLength = 5; + RangeKey recipeKey = RangeKey.create(testRecipePrefix, Collections.singleton(testRecipeLength)); + + MigrationReport migratedNums = job.getMigrationReportForRecipe(recipeKey) + .orElseThrow(() -> new RuntimeException("Migration was expected but from found")); + + assertThat(migratedNums.getValidMigrations().stream().map(MigrationResult::getMigratedNumber) + .collect(Collectors.toList())) + .containsExactly(DigitSequence.of(migratedNumber)); + } + + @Test + public void standardMigration_invalidNumberNoRecipe_expectNoMigration() throws IOException { + String invalidNumber = "1234567"; + MigrationJob job = MigrationFactory.createMigration(invalidNumber, COUNTRY_CODE); + + MigrationReport report = job.getMigrationReportForCountry(); + assertThat(report.getValidUntouchedEntries()).isEmpty(); + assertThat(report.getUntouchedEntries().stream().map(MigrationEntry::getOriginalNumber)) + .containsExactly(invalidNumber); + } + + @Test + public void standardMigration_numberAlreadyValid_expectNoMigration() throws IOException { + String alreadyValidNumber = "84701234567"; + String vietnamCode = "84"; + MigrationJob job = MigrationFactory.createMigration(alreadyValidNumber, vietnamCode); + + MigrationReport report = job.getMigrationReportForCountry(); + assertThat(report.getValidMigrations()).isEmpty(); + assertThat(report.getValidUntouchedEntries().stream().map(MigrationEntry::getOriginalNumber)) + .containsExactly(alreadyValidNumber); + } + + @Test + public void standardMigration_migratableNumber_expectMigration() throws IOException { + String alreadyValidNumber = "841201234567"; + String vietnamCode = "84"; + MigrationJob job = MigrationFactory.createMigration(alreadyValidNumber, vietnamCode); + + MigrationReport report = job.getMigrationReportForCountry(); + assertThat(report.getValidMigrations().stream().map(res -> res.getMigrationEntry().getOriginalNumber())) + .containsExactly(alreadyValidNumber); + } + + @Test + public void standardMigration_invalidMigration_expectInvalidMigration() throws IOException { + DigitSequence migratingNumber = DigitSequence.of("12345"); + MigrationJob job = createMockJobFromTestRecipes(migratingNumber, DigitSequence.of(COUNTRY_CODE), + false); + + MigrationReport report = job.getMigrationReportForCountry(); + assertThat(report.getValidMigrations()).isEmpty(); + assertThat(report.getInvalidMigrations().stream() + .map(result -> result.getMigrationEntry().getOriginalNumber())) + .containsExactly(migratingNumber.toString()); + } + + @Test + public void invalidMigration_strictExport_expectFileWithOriginalNumber() throws IOException { + DigitSequence migratingNumber = DigitSequence.of("12345"); + boolean exportInvalidMigrations = false; + + MigrationJob job = createMockJobFromTestRecipes(migratingNumber, DigitSequence.of(COUNTRY_CODE), + exportInvalidMigrations); + MigrationReport report = job.getMigrationReportForCountry(); + + Path createdFileLocation = Paths.get(report.exportToFile("strictTestFile")); + List createdFileContent = Files.readAllLines(createdFileLocation); + Files.delete(createdFileLocation); + + assertThat(createdFileContent).containsExactly(migratingNumber.toString()); + } + + @Test + public void invalidMigration_exportInvalidMigrations_expectFileWithMigratedNumber() throws IOException { + DigitSequence migratingNumber = DigitSequence.of("12345"); + // what the number is converted to after the matching recipe from testRecipesFile.csv is applied + String numberAfterMigration = "+213456"; + boolean exportInvalidMigrations = true; + + MigrationJob job = createMockJobFromTestRecipes(migratingNumber, DigitSequence.of(COUNTRY_CODE), + exportInvalidMigrations); + MigrationReport report = job.getMigrationReportForCountry(); + + Path createdFileLocation = Paths.get(report.exportToFile("lenientTestFile")); + List createdFileContent = Files.readAllLines(createdFileLocation); + Files.delete(createdFileLocation); + + assertThat(createdFileContent).contains(numberAfterMigration); + } + + @Test + public void standardMigrations_testingEveryInternalRecipe_expectValidMigrations() throws IOException { + ImmutableMap> sampleNumberLists = getSampleNumberListsForAllMigratableCountries(); + MigrationJob job; + + for (String countryCode : sampleNumberLists.keySet()) { + job = MigrationFactory.createMigration(sampleNumberLists.get(countryCode), countryCode, /* exportInvalidMigrations= */ false); + MigrationReport report = job.getMigrationReportForCountry(); + + try { + assertThat(report.getValidMigrations()).isNotEmpty(); + assertThat(report.getInvalidMigrations()).isEmpty(); + assertThat(report.getUntouchedEntries()).isEmpty(); + } catch (AssertionError e) { + Assert.fail("Regarding country code '" + countryCode + "' recipes\n\n" + e.getMessage()); + } + } + } + + /** + * Creates a mock standard number migration from the test recipes csv file. Allows testing of standard (non-custom) + * migrations. + */ + private MigrationJob createMockJobFromTestRecipes(DigitSequence migratingNumber, + DigitSequence countryCode, + boolean exportInvalidMigrations) throws IOException { + + Path recipesPath = Paths.get(TEST_DATA_PATH + "testRecipesFile.csv"); + ImmutableList numberRanges = ImmutableList + .of(MigrationEntry.create(migratingNumber, migratingNumber.toString())); + CsvTable recipes = MigrationFactory + .importRecipes(Files.newInputStream(recipesPath)); + + MetadataZipFileReader metadata = MetadataZipFileReader.of(MigrationFactory.class + .getResourceAsStream(MigrationFactory.METADATA_ZIPFILE)); + CsvTable ranges = metadata.importCsvTable(DigitSequence.of(COUNTRY_CODE)) + .orElseThrow(RuntimeException::new); + + return new MigrationJob(numberRanges, countryCode, recipes, ranges, exportInvalidMigrations); + } + + /** + * Traverses through the internal recipes.csv file and creates sample number lists for every country code that has an + * available migration recipe. A single recipe can be used for a varying range of numbers so for each recipe, the + * smallest number it can migrate and the greatest number it can migrate are added to the given country code list. + */ + private ImmutableMap> getSampleNumberListsForAllMigratableCountries() throws IOException { + ImmutableMap.Builder> sampleNumberLists = ImmutableMap.builder(); + CsvTable recipes = MigrationFactory.importRecipes(MigrationFactory.class + .getResourceAsStream(MigrationFactory.DEFAULT_RECIPES_FILE)); + + RangeTree ranges; + for (DigitSequence countryCode : recipes.getValues(RecipesTableSchema.COUNTRY_CODE)) { + ImmutableList.Builder numberList = ImmutableList.builder(); + ranges = RecipesTableSchema.toRangeTable(recipes).getRanges(RecipesTableSchema.COUNTRY_CODE, countryCode); + + ranges.asRangeSpecifications().forEach(rangeSpecification -> { + numberList.add(rangeSpecification.min().toString()); + numberList.add(rangeSpecification.max().toString()); + }); + sampleNumberLists.put(countryCode.toString(), numberList.build()); + } + return sampleNumberLists.build(); + } +} diff --git a/migrator/src/test/java/com/google/phonenumbers/migrator/MigrationUtilsTest.java b/migrator/src/test/java/com/google/phonenumbers/migrator/MigrationUtilsTest.java new file mode 100644 index 000000000..40c458a7d --- /dev/null +++ b/migrator/src/test/java/com/google/phonenumbers/migrator/MigrationUtilsTest.java @@ -0,0 +1,131 @@ +/* + * Copyright (C) 2020 The Libphonenumber Authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.phonenumbers.migrator; + +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.truth.Truth8.assertThat; + +import com.google.common.collect.ImmutableMap; +import com.google.i18n.phonenumbers.metadata.DigitSequence; +import com.google.i18n.phonenumbers.metadata.RangeSpecification; +import com.google.i18n.phonenumbers.metadata.table.Column; +import com.google.i18n.phonenumbers.metadata.table.CsvTable; +import com.google.i18n.phonenumbers.metadata.table.RangeKey; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.Collections; +import java.util.Optional; +import java.util.stream.Collectors; +import java.util.stream.Stream; +import org.junit.Assert; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.JUnit4; + +@RunWith(JUnit4.class) +public class MigrationUtilsTest { + + private static final String TEST_DATA_PATH = "./src/test/java/com/google/phonenumbers/migrator/testing/testData/"; + private static final Path RECIPES_PATH = Paths.get(TEST_DATA_PATH + "testRecipesFile.csv"); + private static final String COUNTRY_CODE = "44"; + private static final String VALID_TEST_NUMBER = "123"; + + @Test + public void getCountryMigratableNumbers_expectNoMatches() throws IOException { + String invalidTestNumber = "34"; + MigrationJob job = MigrationFactory.createCustomRecipeMigration(invalidTestNumber, COUNTRY_CODE, MigrationFactory + .importRecipes(Files.newInputStream(RECIPES_PATH))); + + Stream noMatchesRange = MigrationUtils + .getMigratableRangeByCountry(job.getRecipesRangeTable(), job.getCountryCode(), + job.getMigrationEntries()); + assertThat(noMatchesRange.collect(Collectors.toSet())).isEmpty(); + } + + @Test + public void getCountryMigratableNumbers_expectMatches() throws IOException { + String numbersPath = TEST_DATA_PATH + "testNumbersFile.txt"; + MigrationJob job = MigrationFactory + .createCustomRecipeMigration(Paths.get(numbersPath), COUNTRY_CODE, MigrationFactory + .importRecipes(Files.newInputStream(RECIPES_PATH))); + + Stream matchesRange = MigrationUtils + .getMigratableRangeByCountry(job.getRecipesRangeTable(), job.getCountryCode(), + job.getMigrationEntries()); + + assertThat(matchesRange.collect(Collectors.toList())) + .containsExactlyElementsIn(job.getMigrationEntries().collect(Collectors.toList())); + } + + @Test + public void getMigratableNumbers_invalidKey_expectException() throws IOException { + RangeSpecification testRangeSpec = RangeSpecification.from(DigitSequence.of(VALID_TEST_NUMBER)); + RangeKey invalidKey = RangeKey.create(testRangeSpec, Collections.singleton(3)); + + MigrationJob job = MigrationFactory.createCustomRecipeMigration(VALID_TEST_NUMBER, COUNTRY_CODE, MigrationFactory + .importRecipes(Files.newInputStream(RECIPES_PATH))); + try { + MigrationUtils + .getMigratableRangeByRecipe(job.getRecipesCsvTable(), invalidKey, job.getMigrationEntries()); + Assert.fail("Expected RuntimeException and did not receive"); + } catch (RuntimeException e) { + assertThat(e).isInstanceOf(IllegalArgumentException.class); + assertThat(e).hasMessageThat().contains(invalidKey.toString()); + } + } + + @Test + public void getMigratableNumbers_validKey_expectNoExceptionAndNoMatches() throws IOException { + RangeSpecification testRangeSpec = RangeSpecification.from(DigitSequence.of("12")); + RangeKey validKey = RangeKey.create(testRangeSpec, Collections.singleton(5)); + + MigrationJob job = MigrationFactory.createCustomRecipeMigration(VALID_TEST_NUMBER, COUNTRY_CODE, MigrationFactory + .importRecipes(Files.newInputStream(RECIPES_PATH))); + + assertThat(MigrationUtils + .getMigratableRangeByRecipe(job.getRecipesCsvTable(), validKey, job.getMigrationEntries()) + .collect(Collectors.toSet())) + .isEmpty(); + } + + @Test + public void findMatchingRecipe_expectNoMatchingRecipe() throws IOException { + MigrationJob job = MigrationFactory.createCustomRecipeMigration(VALID_TEST_NUMBER, COUNTRY_CODE, MigrationFactory + .importRecipes(Files.newInputStream(RECIPES_PATH))); + + DigitSequence testNumberToMatch = DigitSequence.of("12"); + assertThat(MigrationUtils + .findMatchingRecipe(job.getRecipesCsvTable(), job.getCountryCode(), testNumberToMatch)) + .isEmpty(); + } + + @Test + public void findMatchingRecipe_expectMatchingRecipe() throws IOException { + MigrationJob job = MigrationFactory.createCustomRecipeMigration(VALID_TEST_NUMBER, COUNTRY_CODE, MigrationFactory + .importRecipes(Files.newInputStream(RECIPES_PATH))); + DigitSequence testNumberToMatch = DigitSequence.of("12345"); + + Optional, Object>> foundRecipe = MigrationUtils + .findMatchingRecipe(job.getRecipesCsvTable(), job.getCountryCode(), testNumberToMatch); + assertThat(foundRecipe).isPresent(); + + RangeSpecification oldFormat = RangeSpecification + .parse((String) foundRecipe.get().get(RecipesTableSchema.OLD_FORMAT)); + assertThat(oldFormat.matches(testNumberToMatch)).isTrue(); + } +} diff --git a/migrator/src/test/java/com/google/phonenumbers/migrator/testing/testData/testMetadataZip.zip b/migrator/src/test/java/com/google/phonenumbers/migrator/testing/testData/testMetadataZip.zip new file mode 100644 index 000000000..64eda45e1 Binary files /dev/null and b/migrator/src/test/java/com/google/phonenumbers/migrator/testing/testData/testMetadataZip.zip differ diff --git a/migrator/src/test/java/com/google/phonenumbers/migrator/testing/testData/testNumbersFile.txt b/migrator/src/test/java/com/google/phonenumbers/migrator/testing/testData/testNumbersFile.txt new file mode 100644 index 000000000..6b510d406 --- /dev/null +++ b/migrator/src/test/java/com/google/phonenumbers/migrator/testing/testData/testNumbersFile.txt @@ -0,0 +1,5 @@ ++12344 ++12 2 1 2 +126 89 ++12345 +11321 \ No newline at end of file diff --git a/migrator/src/test/java/com/google/phonenumbers/migrator/testing/testData/testRecipesFile.csv b/migrator/src/test/java/com/google/phonenumbers/migrator/testing/testData/testRecipesFile.csv new file mode 100644 index 000000000..f7591d573 --- /dev/null +++ b/migrator/src/test/java/com/google/phonenumbers/migrator/testing/testData/testRecipesFile.csv @@ -0,0 +1,7 @@ +Old Prefix; Old Length; Country Code; Old Format; New Format; Is Final Migration; Description +12; 5; 44; 12xxx; 21xxx6; true; add and replace digits -- valid +99; 3; 84; 9xx; 3xx1; true; add 1 to end of numbers -- valid +13; 5; 44; 12xxx; xxx6; false; Old Format not in key range -- invalid +11; 5; 44; 1xxxx; xx0xx1; true; add and subtract digits -- valid +10; 5; 44; 10xxx; xxx77; false; non final migration with no next recipe -- invalid +15; 5; 44; x5xxx; x1xxx; false; multiple migration linking to (11, 5) -- valid diff --git a/migrator/src/test/java/com/google/phonenumbers/migrator/testing/testUtils/AssertUtil.java b/migrator/src/test/java/com/google/phonenumbers/migrator/testing/testUtils/AssertUtil.java new file mode 100644 index 000000000..bbecacb3e --- /dev/null +++ b/migrator/src/test/java/com/google/phonenumbers/migrator/testing/testUtils/AssertUtil.java @@ -0,0 +1,22 @@ +package com.google.phonenumbers.migrator.testing.testUtils; + +/** Additional useful assertion methods for testing. */ +public final class AssertUtil { + + /** Asserts the given code threw the expected exception, and returns it for further checking. */ + public static T assertThrows(Class clazz, Runnable fn) { + String message; + try { + fn.run(); + message = String.format("expected exception (%s) was not thrown", clazz.getSimpleName()); + } catch (Throwable t) { + if (clazz.isInstance(t)) { + return clazz.cast(t); + } + message = String.format("expected (%s), but caught: %s", clazz.getSimpleName(), t); + } + throw new AssertionError(message); + } + + private AssertUtil() {} +}