Change large math/big test literals to hex.

In September 2022, the Python team addressed a possible DoS issue converting big integers to and from base 10 strings: https://github.com/python/cpython/issues/95778

Converting to/from base 10 is a quadratic operation, so they limited it to 4300 digits: https://discuss.python.org/t/int-str-conversions-broken-in-latest-python-bugfix-releases/18889/83

Github CI still uses an old Python version which parsed our test suite just fine. This patch converts them to hex literals to ensure our test doesn't break when Github does update to a non-vulnerable Python version released after September 2022.

tests/core/.gitignore

@@ -1,2 +1,3 @@
 *.zip
-*.png
+*.png
+math_big_test_library.*