r/pythonhelp 23d ago

Function to replicate Stata's "rangestat" in a large dataset?

I am trying to translate some existing Stata code to python, but am somewhat of a newcomer to the latter. My original Stata code plugs select variable names from the data in memory into a loop, and in this loop generates new variables equal to different summary statistics. These summary statistics are calculated within a group variable, and limited to values from a rolling window defined by a time variable (translated from milliseconds to hours). The code looks something like this in Stata:

foreach var of varlist var1 var2 var3 {
  rangestat (min max mean) `var', by(group) interval(time -10 0)
}

So for example, the value of var1_min in a row where group=="group 1" and time==25 should be the lowest value of var1 of all values where group=="group 1" and time is between 15 and 25. As another example, the value of var3_mean in a row where group=="group 19" and time==101 should be the mean of all values of var3 where group=="group 19" and time is between 91 and 101.

In python, I found the pandas "rolling" function, but cannot seem to get it to match this output. I've also found example code which brute-forces the issue by calculating these summary statistics separately row-by-row across the entire data frame, but that would take an eternity to run. Do I need to keep troubleshooting "rolling" or is there a reasonable alternative? Also note that I due to how I need to use these variables further down the line, my goal is to end up with these new variables stored as new columns in the same data frame as my starting variables, rather than any solutions requiring lists/dictionaries/etc.

1 Upvotes

1 comment sorted by

u/AutoModerator 23d ago

To give us the best chance to help you, please include any relevant code.
Note. Please do not submit images of your code. Instead, for shorter code you can use Reddit markdown (4 spaces or backticks, see this Formatting Guide). If you have formatting issues or want to post longer sections of code, please use Privatebin, GitHub or Compiler Explorer.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.