I'm writing an algorithm to predict global average temperatures. I read an academic paper that used tree GP to solve the same problem. I already have a linear GP algorithm I wrote for something else that I'd like to recycle, but I'd also like to get the best performance possible out of the temperature algorithm.
Is there an advantage to using tree GP vs. linear GP and vise versa? Are there certain types of problems that are better suited to one representation over the other?
I wrote a small library to evolve Flappy Bird through LGP. Made a small blog post and video about it. You can just skip to the video if you want to see results. Would love feedback on the blog, the study, etc...
I have 4k virtual CPUs which are running different op codes per instruction (in practice, a switch using a byte). If I run the warps in sequence by thread ID = CPU ID, then I have potentially all different pathways and I get a performance hit from thread divergence.
I have considered to instead use a layer of indirection where the threads would use a table to point to the virtual CPU to be run (thread ID -> lookup table has index/offset to CPU data), where they are grouped by next instruction value - removing the branching problem for most warps. However, it's unclear if they can be sorted efficiently enough for this to pay off.
Is it possible there is a multi-threaded sorting method that would be fast enough to justify sorting by next instruction? Perhaps a method of pre-fetching the op code bytes for the next instructions and running the logic using the fast register memory? Perhaps some kind of pre-processing is needed rather than doing this as it's running?
Suppose we have 4096 virtual CPUs with op codes 1 byte in size, which is used in a switch statement to perform the actual operation on the vCPUs register file. In the case that mutations are applied to the op codes regularly, it is unlikely that all of the threads in a thread block (each running one vCPU) will have the same op codes at their current instruction pointer. This leads to divergence as I understand it, where all threads must wait while a subset of threads perform a single op code; in the worst case all different switch blocks are executed.
I had considered: what if we group threads together which are executing the same op code? If we used block shared memory (which is fast) to store 4096 bytes - the next op code for each of these vCPUs - can we quickly sort groups of 32 (or whatever block size need be) vCPU indexes (really just 4096 large structs contiguous in memory) so that for example threads 0...31 point to and execute vCPUs all with the same op code (the actual vCPU indexes will be in no particular order, but must be assigned once and only once), and so on that the majority of thread blocks run all the same op code within the block with no divergence, and then a few blocks at the end will run the remainder (slower, many op codes, but overall not much of a performance impact)?
The sorting and selection would have to work in the threaded environment, and I can't think of anything right now to do it in a thread safe manner.
Found a couple of papers related to divergence: http://www.cs.ucr.edu/~gupta/research/Publications/Comp/micro-48.pdf , https://arxiv.org/pdf/1504.01650.pdf
I know this question isn’t specifically related to data science (perhaps in the broad term), but for those coming from OR, why hasn’t there been a rise in linear programming and optimization as it relates to analytics and BI? Is this more in line with decision science?
I made a 2 minute intro video to Linear programming.
https://www.youtube.com/watch?v=C0TTxV0n9OA
Most people in ML probably start directly with complicated models like Neural Networks, but as is often the case, a simple model (e.g., a linear one) is much appropriate, and if not, then at least it gives a good baseline for all the future complications your want to try.
Linear Programming is arguably the most useful method is all of mathematical optimization, and there is a big ongoing effort to improve solve methods. This video illustrate the two main methods for doing that: the Simplex method that jumps for one vertex to the next hoping to improve the solution, and the interior point method that goes through the feasible set towards the optimal solution.
I remember taking an Operations Research course using these models. I enjoyed the course and like the concept/purpose, but I'm curious to know if it's worth learning in greater depth.
I wrote a small library to evolve Flappy Bird through LGP. Made a small blog post and video about it. You can just skip to the video if you want to see results. Would love feedback on the blog, the study, etc...
