## Hacking apgsearch

For scripts to aid with computation or simulation in cellular automata.
muzik
Posts: 3925
Joined: January 28th, 2016, 2:47 pm
Location: Scotland

### Re: Hacking apgsearch

Could you mod apgsearch 3.x?
Bored of using the Moore neighbourhood for everything? Introducing the Range-2 von Neumann isotropic non-totalistic rulespace!

A for awesome
Posts: 2160
Joined: September 13th, 2014, 5:36 pm
Location: Pembina University, Home of the Gliders
Contact:

### Re: Hacking apgsearch

muzik wrote:Could you mod apgsearch 3.x?
In theory, I think so. In practice, I don't know enough of the low-level and hardware-specific tricks used in 3.x to do that, and it doesn't even compile using any compiler I can figure out how to get on my old Mac. It would just be far easier for me to write something completely from scratch. Anyone that knows how, though, please feel free to make a mod of 3.x (assuming it's actually possible and there's not some stupid hang-up I'm missing).
praosylen#5847 (Discord)

x₁=ηx
V*_η=c²√(Λη)
K=(Λu²)/2
Pₐ=1−1/(∫^∞_t₀(p(t)ˡ⁽ᵗ⁾)dt)

$$x_1=\eta x$$
$$V^*_\eta=c^2\sqrt{\Lambda\eta}$$
$$K=\frac{\Lambda u^2}2$$
$$P_a=1-\frac1{\int^\infty_{t_0}p(t)^{l(t)}dt}$$

Scorbie
Posts: 1485
Joined: December 7th, 2013, 1:05 am

### Re: Hacking apgsearch

muzik wrote:I like asking stupid questions:

Could you mod apgsearch 3.x?
I thought semitotalistoc rules are supported by 3.x, no? I **think** that it is possible by merely tweaking rule2asm.py..
Best wishes to you, Scorbie

muzik
Posts: 3925
Joined: January 28th, 2016, 2:47 pm
Location: Scotland

### Re: Hacking apgsearch

Scorbie wrote:
muzik wrote:I like asking stupid questions:

Could you mod apgsearch 3.x?
I thought semitotalistoc rules are supported by 3.x, no? I **think** that it is possible by merely tweaking rule2asm.py..
not that I know how to do that.

I haven't tried searching non totalistic rules with it, but checking hauls submitted for them shows that pretty much everyone uses the python version.
Bored of using the Moore neighbourhood for everything? Introducing the Range-2 von Neumann isotropic non-totalistic rulespace!

A for awesome
Posts: 2160
Joined: September 13th, 2014, 5:36 pm
Location: Pembina University, Home of the Gliders
Contact:

### Re: Hacking apgsearch

Scorbie wrote:I thought semitotalistoc rules are supported by 3.x, no? I **think** that it is possible by merely tweaking rule2asm.py..
They aren't; apparently the boolean circuit corresponding to some rules ended up containing too many gates or something (I don't really understand this part especially well), so it was decided against. Tweaking rule2asm.py might be possible, but I don't know asm, so I'm not the one to test it, especially on my particular old computer. There may also be some other sections of the script (I would guess at least pseudo-object separation) that require knowledge of the particular rule, so those would take more work to mod.
praosylen#5847 (Discord)

x₁=ηx
V*_η=c²√(Λη)
K=(Λu²)/2
Pₐ=1−1/(∫^∞_t₀(p(t)ˡ⁽ᵗ⁾)dt)

$$x_1=\eta x$$
$$V^*_\eta=c^2\sqrt{\Lambda\eta}$$
$$K=\frac{\Lambda u^2}2$$
$$P_a=1-\frac1{\int^\infty_{t_0}p(t)^{l(t)}dt}$$

drc
Posts: 1664
Joined: December 3rd, 2015, 4:11 pm
Location: creating useless things in OCA

### Re: Hacking apgsearch

Can anybody have apgmera print every new object that appears:

New object: xp2_7
New object: xs4_33
New object: xq4_153
.
.
.
10000 soups completed (99999999 per second)
\100\97\110\105

Apple Bottom
Posts: 1034
Joined: July 27th, 2015, 2:06 pm
Contact:

### Re: Hacking apgsearch

drc wrote:Can anybody have apgmera print every new object that appears
Wrong thread? I think this one's really for apgsearch 1.x, not apg(mera|nano). That said: patch here.
If you speak, your speech must be better than your silence would have been. — Arabian proverb

Catagolue: Apple Bottom • Life Wiki: Apple Bottom • Twitter: @_AppleBottom_

Proud member of the Pattern Raiders!

wildmyron
Posts: 1488
Joined: August 9th, 2013, 12:45 am
Location: Western Australia

### Re: Hacking apgsearch

A for awesome wrote:
wildmyron wrote:
drc wrote:It's having a hard time searching B2e3/S23-j, an extremely stable rule. What's going on (and I said heyyeyaaeyaaaeyaeyaa)
I think you meant to post this on the Hacking apgsearch thread, but in any case something is going wrong with the rule parsing. When I tried a search with apgsearch-2016-2-06-v0.54+0.21i-update.py it was actually trying to search B23/S23 - doomed to failure. Searching works fine when I used my version of non-totalistic apgsearch (apgsearch-isotropic-v0.2.py) The search runs as expected. (No upload capability though.) I have no idea what's going wrong there.

Btw, that's a nice rule, lots of variety in the natural spaceships.
Did either of you generate the rule table before searching the rule? This is a drawback of 0.21i: it does not autogenerate the rule table for you; you have to use isotropic-rulegen.py to make the rule and enter it using an underscore, not a slash. I'll try to improve this when I get back from vacation in three weeks, or sooner if I can.
I had generated the rule, in particular it was generated by apgsearch-isotropic-v0.2.py while testing the rule. However, I specified the rule as "B2e3/S23-j" to both programs, not "B2e3_S23-j" as you say is required. This works fine in my script and I'd forgotten that the other script requires the exact name of an existing rule file. When I use the '_' the correct rule is processed but like @drc, soup search is very slow (~3 soups/s, increasing to ~7 soups/s if I reduce the sqrtspp_optimal value to 6.

The reason for this is down to the way apgsearch does periodicity detection and the length of time the Coalesce rule is run to ensure high period objects are correctly identified. In this case, p8 and p16 objects are relatively common, but the default periods which are detected by the naive stabilisation detection only include factors of 12 and 30. So even though most soups stabilise quickly, this is not detected in soups with a p8 or p16 object.* If one soup in the page has such an object, it is implicitly assumed there may be a high period object and the coalesce rule is run for much longer than necessary. This is not a big issue in terms of evolving the soup in the auxiliary rules, but all the ships leave a long trail of "dead but has been alive" cells. I believe the slowdown occurs because the final population of the universe before object identification occurs is very large, and everyone of those "dead but has been alive" cells is grabbed by the gridobj() function as part of the ship, as well as being scanned after the object has been removed. From my previous experience this explanation doesn't quite seem to explain the drastic slowdown, but when I make the following change to stabilise3(),

Code: Select all

        # Phase I of stabilisation detection, designed to weed out patterns
# that stabilise into a cluster of low-period oscillators within

if (self.naivestab2(12, 10)):
return 4;

if (self.naivestab(16, 30, 200)):
return 4;

if (self.naivestab(32, 30, 200)):
return 5;
i.e. change the periods given to naivestab() from 12 to 16 and from 30 to 32, then the search speed jumps to over 150 soups/s.

I was running a test set to upload which I was going to cancel because of the plethora of un-separated small gliders, but accidentally uploaded it. At least there's a nice p16 puffer to compensate for all the noise in the p2 ships category.
The latest version of the 5S Project contains over 226,000 spaceships. There is also a GitHub mirror of the collection. Tabulated pages up to period 160 (out of date) are available on the LifeWiki.

drc
Posts: 1664
Joined: December 3rd, 2015, 4:11 pm
Location: creating useless things in OCA

### Re: Hacking apgsearch

Whenever I try to search rules (i.e. B34et/S23-a4eit6) in the neighbourhood of CommonRepl, it always messes with my computer. It'll be going smoothly, then it will hit a pattern like this:

Code: Select all

x = 3, y = 7, rule = B34et_S23-a4eit6
b2o$2bo$b2o2$3o$2bo$b2o!  Then my ram explodes and chrome (playing youtube videos in the background) crashes. I don't know why this is. \100\97\110\105 Apple Bottom Posts: 1034 Joined: July 27th, 2015, 2:06 pm Contact: ### Re: Hacking apgsearch drc wrote:Whenever I try to search rules (i.e. B34et/S23-a4eit6) in the neighbourhood of CommonRepl, it always messes with my computer. It'll be going smoothly, then it will hit a pattern like this: Code: Select all x = 3, y = 7, rule = B34et_S23-a4eit6 b2o$2bo$b2o2$3o$2bo$b2o!

Then my ram explodes and chrome (playing youtube videos in the background) crashes. I don't know why this is.
A random stab in the dark: what are your memory limits in Golly (in File->Preferences->Control)? Might be that you're running out of memory and the OOM killer is kicking in.
If you speak, your speech must be better than your silence would have been. — Arabian proverb

Catagolue: Apple Bottom • Life Wiki: Apple Bottom • Twitter: @_AppleBottom_

Proud member of the Pattern Raiders!

Gamedziner
Posts: 795
Joined: May 30th, 2016, 8:47 pm
Location: Milky Way Galaxy: Planet Earth

### Re: Hacking apgsearch

drc wrote:Whenever I try to search rules (i.e. B34et/S23-a4eit6) in the neighbourhood of CommonRepl, it always messes with my computer. It'll be going smoothly, then it will hit a pattern like this:

Code: Select all

x = 3, y = 7, rule = B34et_S23-a4eit6
b2o$2bo$b2o2$3o$2bo$b2o!  Then my ram explodes and chrome (playing youtube videos in the background) crashes. I don't know why this is. Is it just me, or are those tiny Knightships? Anyways, your computer may just be having trouble with infinite growth. Code: Select all x = 81, y = 96, rule = LifeHistory 58.2A$58.2A3$59.2A17.2A$59.2A17.2A3$79.2A$79.2A2$57.A$56.A$56.3A4$27.
A$27.A.A$27.2A21$3.2A$3.2A2.2A$7.2A18$7.2A$7.2A2.2A$11.2A11$2A$2A2.2A
$4.2A18$4.2A$4.2A2.2A$8.2A!


Apple Bottom
Posts: 1034
Joined: July 27th, 2015, 2:06 pm
Contact:

### Re: Hacking apgsearch

I just tried B34et/S23-a4eit6 myself, and hit one of those soups after about 2000 soups. The culprit is the error-correcting phase in apgsearch; Golly's configured memory limits don't apply, likely because the allocations are done in Python as opposed to as part of Golly's own algorithms.

Golly kept hovering somewhere between 16 and 17 GB of allocated memory here, but did not go beyond that, and the machine otherwise remained usable. I'll leave it running for a while to see whether it's going to go anywhere, but it's quite slow as well (error correction's took almost 25 minutes), so whether it'll ever finish 10k soups and submit results to Catagolue remains to be seen. (It's already hit a similar pattern again almost right after.)

The soup in question's this, BTW:

Code: Select all

x = 16, y = 16, rule = B34et/S23-a4eit6
obbbbbobboooobob$oobbobobbbobobbo$
boooobbbobboobob$bbooboobbbbobobo$
obbooboboboobbbb$bbbboobbbboboboo$
obbooobbobboobbo$boooooboobbooobo$
bbbbobbooboooobb$ooooobbbbbbooobo$
boobooobooboobbb$boboobbbbbobobbo$
oobbbobbbbobbbbo$obooobbbooooobob$
boobbbbboobobobo$bobbbbboboooooob!  Apparently it produced no less than 15 PATHOLOGICALs, in addition to a zz_LINEAR. If you speak, your speech must be better than your silence would have been. — Arabian proverb Catagolue: Apple Bottom • Life Wiki: Apple Bottom • Twitter: @_AppleBottom_ Proud member of the Pattern Raiders! drc Posts: 1664 Joined: December 3rd, 2015, 4:11 pm Location: creating useless things in OCA ### Re: Hacking apgsearch Apple Bottom wrote:16 and 17 GB I have 4 GB... oh well, it can run most things just fine. \100\97\110\105 Apple Bottom Posts: 1034 Joined: July 27th, 2015, 2:06 pm Contact: ### Re: Hacking apgsearch drc wrote:I have 4 GB... oh well, it can run most things just fine. It gets better; apgsearch hit a soup where two of these patterns interact, and Golly's currently using about 21.5 GB of memory. (That's actually more than I have in this machine, but so far everything's still running fine. Fingers crossed.) EDIT: here's the soup: Code: Select all x = 16, y = 16, rule = B34et/S23-a4eit6 booobobbbbooboob$
bbbooboobobbobob$bboobboboobobboo$
oobooboobbooboob$oooooobboobbbbbo$
bbbobooooboobobb$bobobobobboooooo$
ooboobbooboooobb$bbbobobbbbobbbob$
ooobobobbobboooo$oboooboobobooooo$
ooboobobboobobbo$bobbooobbobbobbb$
obobobbbboboobob$obbboboobooobbob$
oooobbobbbbbbbbo!

Let this run for a while (in HashLife), it's really quite neat.
If you speak, your speech must be better than your silence would have been. — Arabian proverb

Catagolue: Apple Bottom • Life Wiki: Apple Bottom • Twitter: @_AppleBottom_

Proud member of the Pattern Raiders!

muzik
Posts: 3925
Joined: January 28th, 2016, 2:47 pm
Location: Scotland

### Re: Hacking apgsearch

..how do you run non totalistic rules in HashLife?
Bored of using the Moore neighbourhood for everything? Introducing the Range-2 von Neumann isotropic non-totalistic rulespace!

Apple Bottom
Posts: 1034
Joined: July 27th, 2015, 2:06 pm
Contact:

### Re: Hacking apgsearch

muzik wrote:..how do you run non totalistic rules in HashLife?
Good question! I'm honestly not sure. Let me clarify the above saying "run it using hyperspeed", which seems to be using some sort of hash-based algorithm -- the pattern ran to a billion generations (with a population to match) in just a few minutes.

Unrelated side note: Golly's now up to using 25 GiB of memory, and browsing the web on the same machine has become noticeably more laggy. Perhaps I should invest in some more RAM.
If you speak, your speech must be better than your silence would have been. — Arabian proverb

Catagolue: Apple Bottom • Life Wiki: Apple Bottom • Twitter: @_AppleBottom_

Proud member of the Pattern Raiders!

muzik
Posts: 3925
Joined: January 28th, 2016, 2:47 pm
Location: Scotland

### Re: Hacking apgsearch

Does this still work with 2.8?
Bored of using the Moore neighbourhood for everything? Introducing the Range-2 von Neumann isotropic non-totalistic rulespace!

A for awesome
Posts: 2160
Joined: September 13th, 2014, 5:36 pm
Location: Pembina University, Home of the Gliders
Contact:

### Re: Hacking apgsearch

muzik wrote:Does this still work with 2.8?
I don't know, but I doubt it's terribly difficult to fix. Try it, and if it doesn't work, just go through the source code -- you can skip the long dictionary of object names and scores -- and add the line

Code: Select all

g.setalgo("RuleLoader")
before every call to g.setrule as a dirty fix. It may take a while to find them all, but it should work (hopefully) after that.

P.S. You can actually use find and replace to do the same thing, at least on good text editors. Just replace all occurrences of

Code: Select all

g.setrule(
with

Code: Select all

g.setalgo("RuleLoader")
g.setrule(
.
praosylen#5847 (Discord)

x₁=ηx
V*_η=c²√(Λη)
K=(Λu²)/2
Pₐ=1−1/(∫^∞_t₀(p(t)ˡ⁽ᵗ⁾)dt)

$$x_1=\eta x$$
$$V^*_\eta=c^2\sqrt{\Lambda\eta}$$
$$K=\frac{\Lambda u^2}2$$
$$P_a=1-\frac1{\int^\infty_{t_0}p(t)^{l(t)}dt}$$

Saka
Posts: 3608
Joined: June 19th, 2015, 8:50 pm
Location: somewhere
Contact:

### Re: Hacking apgsearch

I'm trying to add size symmetries for APG 1. They are L (32x32), S (8x8), and 1D (1 x 20). How do I do this?
Currently taking a little break, but still hanging around on the Discord server.

wildmyron
Posts: 1488
Joined: August 9th, 2013, 12:45 am
Location: Western Australia

### Re: Hacking apgsearch

Saka wrote:I'm trying to add size symmetries for APG 1. They are L (32x32), S (8x8), and 1D (1 x 20). How do I do this?
Start by modifying the hashsoup function to generate soups with the desired "symmetry" For 32x32 you will need more random bits than SHA256 gives you. SHA512 gives you double, but you still need more. You can do this by calculating multiple hash values from slightly modified versions of the input string, e.g:

Code: Select all

def hashsoup(instring, sym):

s = hashlib.sha512(instring+'a').digest() + hashlib.sha512(instring+'b').digest()

...

For 8x8 you need less. You can just use as much as you need, though obviously it's a bit wasteful. For 1x20 there are only just over a million configurations. You can iterate through them all without using any randomness.

Then you need to generate the soup from the random string. You can do this by adjusting the modular arithmetic for the 16x16 case to suit the new soup size.

The real question is: Why? What are you hoping to find that you otherwise won't?
The latest version of the 5S Project contains over 226,000 spaceships. There is also a GitHub mirror of the collection. Tabulated pages up to period 160 (out of date) are available on the LifeWiki.

Saka
Posts: 3608
Joined: June 19th, 2015, 8:50 pm
Location: somewhere
Contact:

### Re: Hacking apgsearch

@wildmyron This is my first attemp at "real" apg hacking. How do I use rhe script you gave? I dont get it as of now
Currently taking a little break, but still hanging around on the Discord server.

wildmyron
Posts: 1488
Joined: August 9th, 2013, 12:45 am
Location: Western Australia

### Re: Hacking apgsearch

@Saka: Well, I only quoted a small script fragment to show how you can generate the required random string. Have a good look at how the original apgsearch hashsoup function worked. I suggest that you special case the generation of the soup for your custom "symmetries" rather than trying to adapt the hashsoup function in apgsearch v1. There are also quite a few other places where small changes need to be made to accommodate new symmetries. I think it's unlikely anyone other than you will make these adaptations because you still haven't answered the question: Why?
The latest version of the 5S Project contains over 226,000 spaceships. There is also a GitHub mirror of the collection. Tabulated pages up to period 160 (out of date) are available on the LifeWiki.

Saka
Posts: 3608
Joined: June 19th, 2015, 8:50 pm
Location: somewhere
Contact:

### Re: Hacking apgsearch

wildmyron wrote: you still haven't answered the question: Why?
I answered it already: I just want to try this as my first "real" hacking attempt
Currently taking a little break, but still hanging around on the Discord server.

wildmyron
Posts: 1488
Joined: August 9th, 2013, 12:45 am
Location: Western Australia

### Re: Hacking apgsearch

Saka wrote:
wildmyron wrote: you still haven't answered the question: Why?
I answered it already: I just want to try this as my first "real" hacking attempt
Well, in that case: Carry on!

I suggest you start with a standalone version of hashsoup() for each of your desired symmetries. Then you can work on integrating it into the rest of apgsearch.
The latest version of the 5S Project contains over 226,000 spaceships. There is also a GitHub mirror of the collection. Tabulated pages up to period 160 (out of date) are available on the LifeWiki.

Apple Bottom
Posts: 1034
Joined: July 27th, 2015, 2:06 pm
Contact:

### Re: Hacking apgsearch

wildmyron wrote:Start by modifying the hashsoup function to generate soups with the desired "symmetry" For 32x32 you will need more random bits than SHA256 gives you. SHA512 gives you double, but you still need more.
On that note, SHA512 would yield enough for 20x20 soups -- might be interesting to investigate those since that's what Nathaniel's census used.

Alternatively, it would also be precisely enough for 16x32, which would be interesting to investigate for much the same reason that 8x32 is.
If you speak, your speech must be better than your silence would have been. — Arabian proverb

Catagolue: Apple Bottom • Life Wiki: Apple Bottom • Twitter: @_AppleBottom_

Proud member of the Pattern Raiders!