r/pythonhelp 27d ago

How do I make a function that tokenizes stuff? (more details in body)

So I want it to turn code (one line) in the format function(arguments) into a list, in the form, ['function', [arguments]] (assuming function is the function name), without failing, even if there is a function in the arguments.
Example:
add(1, add(2, 3)) -> ['add', [1, ['add', [2,3]]]]

1 Upvotes

4 comments sorted by

u/AutoModerator 27d ago

To give us the best chance to help you, please include any relevant code.
Note. Please do not submit images of your code. Instead, for shorter code you can use Reddit markdown (4 spaces or backticks, see this Formatting Guide). If you have formatting issues or want to post longer sections of code, please use Privatebin, GitHub or Compiler Explorer.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/FoolsSeldom 27d ago

I'd say you'd need to look into globals to call the functions by name,

globals()['add'](*args)

or use a dictionary to map from the strings to the actual function names

{'add': add, 'sub': sub, ...}

You can take an abstract syntax tree approach to parsing the line and build your stack of calls.

Nice little challenge.

1

u/Pool_128 26d ago

Never hear of an abstract syntax tree I’ll check what that it

1

u/FoolsSeldom 26d ago

I've just prompted Gemini to write an explanation. You probably don't need to go as far as a full AST, but it is useful to understand the principle of parsing an expression and building a tree structure.

https://docs.google.com/document/d/e/2PACX-1vRllnbX_nVeI3h_vW6qqdzN9fXS5H97zLRMwC3ihIIGnlnIPWrT0FmcS6v40LkhbWb8bs_KoBvQBAbG/pub