apgsearch v5.0

For general discussion about Conway's Game of Life.
etmoonshade
Posts: 9
Joined: May 8th, 2021, 11:48 pm

Re: apgsearch v5.0

Post by etmoonshade » May 9th, 2021, 9:32 pm

Dylan Chen wrote:
May 9th, 2021, 8:09 pm
etmoonshade wrote:
May 9th, 2021, 12:42 pm
62 cores * 1 virtual machine, 62 cores total: ~40k soups/second (s/s)
15 * 4, 60c: ~55k s/s (I think - didn't keep this data specifically)
7 * 8, 56c: ~75k s/s
So with my specific setup (an Epyc 7302 server at 32c/64t, running Hyper-V)
for your reference, the apgsearch running on my WSL / 4600H (6c12t) could reach 8k in single thread, and the multi-thread could reach 60k in total.
some tiny 1c1G server I rent from cloud (with AVX-512) could reach 9k+. you can check https://catagolue.hatsya.com/haul/b3s23 ... oud_Debian
each different number suffix stands for a 1c1G cloud server.

the haul size wouldn't be problem, it is not linear growing with the soup number. And the stability of Catagolue is fairly sound, it withstand 70 tiny server 'attack' in the size of 0.11billion soups.(haul size 53kB)
the interval I set for C1 is usually 2h+, and G1 is 0.5h.
Yup, I'm noticing that (I have a cheapo Azure VM doing ~8k/s,) but the 7302 has a pretty low clock (2.5GHz,) all things considered - I'm more surprised by how poorly the threads scale when they're on a single VM vs. multiples running on the same host. I get the feeling I have some tweaking to do...

As for the haul counts, thanks for the info - sorry if I'm focusing on this too much, I've just read a lot about people breaking stuff and I'm not 100% sure on the unspoken rules for doing something with multiple machines submitting hauls/etc. :)

Also, that haul suffix idea is clever, and I am totally stealing it. Eventually, next time I stop and rebuild everything. :D

Edit: I have powershell scripts now that do most of the build and setup for me on a VM server - would there be an appropriate place to release/post them (and presumably a second build script for a cloud server?) Would that even be useful for anyone?

Dylan Chen
Posts: 113
Joined: March 27th, 2020, 8:07 am
Contact:

Re: apgsearch v5.0

Post by Dylan Chen » May 9th, 2021, 9:58 pm

etmoonshade wrote:
May 9th, 2021, 9:32 pm
Edit: I have powershell scripts now that do most of the build and setup for me on a VM server - would there be an appropriate place to release/post them (and presumably a second build script for a cloud server?) Would that even be useful for anyone?
maybe the 'Scripts' sub-forum would suit that, it always good to share and backup, just in case.



From my humble experience of soup searching. the benchmark (of search speed) is mainly decided by the OS plantform. Linux > WSL > Cywin > VM. even different distribution would have 10% speed differency. Debian > Ubuntu.

If possible, it is more efficient for multi-thread machine to do spaceship search (rather than soup search). the soup search now can be do with GPU accelerate. G1 now produce tenfold soups than traditional C1.https://catagolue.hatsya.com/census/b3s23/G1
In this morning, a new elementary Knight spaceship has been found. it took 4 months of search and 45k of CPU hours. super mainframes would speed up the search process.
Tools should not be the limit.
Whether your obstacle is a script, an stdin, or Linux environment computing resouces.
check New rules thread for help.

User avatar
calcyman
Posts: 2425
Joined: June 1st, 2009, 4:32 pm

Re: apgsearch v5.0

Post by calcyman » May 9th, 2021, 10:00 pm

etmoonshade wrote:
May 9th, 2021, 9:32 pm
As for the haul counts, thanks for the info - sorry if I'm focusing on this too much, I've just read a lot about people breaking stuff
There's no need to worry about this. Every Catagolue downtime that's happened in the past has been from searching other cellular automata (not B3/S23).
and I'm not 100% sure on the unspoken rules for doing something with multiple machines submitting hauls/etc. :)
There aren't any unspoken rules. :)

No-one will complain if you submit lots of hauls with reasonable sizes (say, 10M -- 100M soups). If you're submitting 100M-soup hauls, then you'd need about 2000 CPU cores running continually to be able to saturate Catagolue's current safety limit of 'process <= 120 hauls every 12-minute update cycle'. And that doesn't actually cause anything to fail; it just means that the queue of unprocessed hauls will steadily grow.
What do you do with ill crystallographers? Take them to the mono-clinic!

etmoonshade
Posts: 9
Joined: May 8th, 2021, 11:48 pm

Re: apgsearch v5.0

Post by etmoonshade » May 9th, 2021, 11:31 pm

Dylan Chen wrote:
May 9th, 2021, 9:58 pm
From my humble experience of soup searching. the benchmark (of search speed) is mainly decided by the OS plantform. Linux > WSL > Cywin > VM. even different distribution would have 10% speed differency. Debian > Ubuntu.
Just a quick little update on this. Because I've been thinking "wow, that's kinda slow," I've been digging around for anything I could do to tune my system. Turns out that Microsoft decided to apply a "balanced" power plan by default to a server OS. I was absolutely livid when I finally figured that out.

On the upside, I went from:

Code: Select all

Overall Average soups/sec: 9217.16925
Total soups/sec: 73737.354
To:

Code: Select all

Overall Average soups/sec: 17471.580625
Total soups/sec: 139772.645
across 8 servers. And now I probably need to re-run tests with higher core counts per machine.

calcyman wrote:
May 9th, 2021, 10:00 pm
There aren't any unspoken rules. :)
And now I know there aren't any unspoken rules. :D
calcyman wrote:
May 9th, 2021, 10:00 pm
No-one will complain if you submit lots of hauls with reasonable sizes (say, 10M -- 100M soups).
That's kind of the guidance I was looking for, ultimately. Thanks again!

User avatar
GUYTU6J
Posts: 1411
Joined: August 5th, 2016, 10:27 am
Location: 拆哪!I repeat, CHINA! (a.k.a. 种花家)
Contact:

Re: apgsearch v5.0

Post by GUYTU6J » May 14th, 2021, 2:16 am

Suppose I'm piping the output RLE's from an external program to apgsearch with the separator "|". When apgseach pauses searching to upload a haul to catagolue, would the external program be paused as well? If not, for the purpose of some enumeration, part of the results will be lost.

If in the case above I have no prior knowledge about the number of results from an enumeration, how should I define the number of soups per haul?
Lifequote:
In the drama The Peony Pavilion, Tang Xianzu wrote: 原来姹紫嫣红开遍,似这般都付与断井颓垣。
(Here multiflorate splendour blooms forlorn
Midst broken fountains, mouldering walls.)
I'm afraid there's arrival but no departure.
Chinese people, enjoy your 🥩🥚🥛 and 🐟!

Dylan Chen
Posts: 113
Joined: March 27th, 2020, 8:07 am
Contact:

Re: apgsearch v5.0

Post by Dylan Chen » May 14th, 2021, 2:54 am

GUYTU6J wrote:
May 14th, 2021, 2:16 am
Suppose I'm piping the output RLE's from an external program to apgsearch with the separator "|".
from the humble experience of running ROR_stdin. the apgsearch would not miss any input, thus we can 'assume' in each soup circle, external program would pause until this very soup got searched.

GUYTU6J wrote:
May 14th, 2021, 2:16 am
how should I define the number of soups per haul?
it may depend on various factors, search speed, search efficiency. btw, the haul size of ROR_stdin is so huge that easily exceed 1000kB, the haul size I set is 10000 (the minimal limit of stdin).
Tools should not be the limit.
Whether your obstacle is a script, an stdin, or Linux environment computing resouces.
check New rules thread for help.

User avatar
GUYTU6J
Posts: 1411
Joined: August 5th, 2016, 10:27 am
Location: 拆哪!I repeat, CHINA! (a.k.a. 种花家)
Contact:

Re: apgsearch v5.0

Post by GUYTU6J » May 14th, 2021, 3:34 am

Dylan Chen wrote:
May 14th, 2021, 2:54 am
GUYTU6J wrote:
May 14th, 2021, 2:16 am
If in the case above I have no prior knowledge about the number of results from an enumeration, how should I define the number of soups per haul?
it may depend on various factors, search speed, search efficiency. btw, the haul size of ROR_stdin is so huge that easily exceed 1000kB, the haul size I set is 10000 (the minimal limit of stdin).
Emm, you're not getting my point. I do not really care about time consumption here. Say the enumeration will yield 10,001 soups containing 25 single object each, and I set 10000 as the number of soups per haul, then the last 25 objects in the last soup will fail to be uploaded. I hope this kind of situation does not happen, but I don't see obvious methods except for uploading objects one by one (which needs bypassing the current minimum size of a haul).

---

Also, what is the current threshold for oversized/pathological patterns, especially linear growth patterns?
Lifequote:
In the drama The Peony Pavilion, Tang Xianzu wrote: 原来姹紫嫣红开遍,似这般都付与断井颓垣。
(Here multiflorate splendour blooms forlorn
Midst broken fountains, mouldering walls.)
I'm afraid there's arrival but no departure.
Chinese people, enjoy your 🥩🥚🥛 and 🐟!

Dylan Chen
Posts: 113
Joined: March 27th, 2020, 8:07 am
Contact:

Re: apgsearch v5.0

Post by Dylan Chen » May 14th, 2021, 4:06 am

GUYTU6J wrote:
May 14th, 2021, 3:34 am
Say the enumeration will yield 10,001 soups containing 25 single object each, and I set 10000 as the number of soups per haul, then the last 25 objects in the last soup will fail to be uploaded.

the method I use is :

Code: Select all

fill_rle="x = 2, y = 2, rule = B3/S23 \n 2o$2o!"
 #block
use blocks to fill the soup limit (reach 10000)
Tools should not be the limit.
Whether your obstacle is a script, an stdin, or Linux environment computing resouces.
check New rules thread for help.

User avatar
LaundryPizza03
Posts: 1319
Joined: December 15th, 2017, 12:05 am
Location: Unidentified location "https://en.wikipedia.org/wiki/Texas"

Re: apgsearch v5.0

Post by LaundryPizza03 » May 17th, 2021, 11:19 am

With the latest version of apgsearch, I got a strange error when trying to run a B0 rule:

Code: Select all

Rule b0136s0124 cannot be iterated 1 x 2^0 generations.

Code: Select all

x = 4, y = 3, rule = B3-q4z5y/S234k5j
2b2o$b2o$2o!
LaundryPizza03 at Wikipedia

The latest edition of new-gliders.db.txt and oscillators.db.txt have 31531 spaceships and 1293 oscillators from outer-totalistic rules. You are invited to help!

User avatar
yujh
Posts: 2285
Joined: February 27th, 2020, 11:23 pm
Location: 我不觉得我迷路了,我可能在K2-146 b上 (@bibunsekibun)
Contact:

Re: apgsearch v5.0

Post by yujh » Yesterday, 5:56 am

Excuse me, is there a place to tell me how to search for other types of rules like LTL?
B34kz5e7c8/S23-a4ityz5k!!!

b2n3-q5y6cn7s23-k4c8

B3-kq6cn8/S2-i3-a4ciyz8

wiki

Rule modifier

Catagolue stucks

Code: Select all

ping catagolue.hatsya.com
Pinging ghs.googlehosted.com [172.217.24.19]
Ping statistics for 172.217.24.19:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss)
中考全优秀

Post Reply