Implement Tokenize Function

A basic string splitter is provided that breaks a string into tokens according to a given set of delimiter characters. The baseline implementation checks each character against every delimiter, leading to poor performance on large inputs. You are to reimplement the splitting logic so that it completes all test suites in under ten seconds. The output of your version must be identical to that of the baseline routine. To meet the performance requirement, your solution must make…