apgsearch v2.2

[Alexey_Nigin]

I have checked whether Catagolue is vulnerable to SQL injection.

Result: vulnerable.

Description of vulnerability: the chart breaks if the username ends with backslash (\).

Page I tried this on: b3s23/D2_+1.

Suggested solution: replace each backslash with double backslash (\\). Also, you replace each apostrophe (') with ('), which seems kinda clumsy to me. Don't you think it would be better to use backslash+apostrophe (\')?

Note to everybody: trying this on other pages before this is fixed is NOT a funny prank.

[calcyman]

I have checked whether Catagolue is vulnerable to SQL injection.

Result: vulnerable.

Description of vulnerability: the chart breaks if the username ends with backslash (\).

Catagolue doesn't use SQL; the so-called 'vulnerability' is just a client-side display error. And you can't do a js injection since quotation marks are sanitised.

I've now resolved the problem by the means of the following line of code:

Code: Select all

String displayedName = parts[1].replaceAll("'","'").replaceAll("\\\\","\\\\\\\\");

Amusingly, it requires eight backslashes in the replaced string because:

• Backslashes need to be escaped in string literals;
• Backslashes need to be escaped in the regex-style replaceAll syntax;
• Backslashes need to be escaped in the generated client-side js.

[SuperJedi224]

There should be a top 100 most common patterns page for each rule in the catalogue. There should also be a page explaining the apgcode syntax.

[calcyman]

There should be a top 100 most common patterns page for each rule in the catalogue.

You mean like this? https://catagolue.appspot.com/textcensus/b3s23/C1

There should also be a page explaining the apgcode syntax.

You mean like this? https://catagolue.appspot.com/help/wechsler.txt

[SuperJedi224]

There should be a top 100 most common patterns page for each rule in the catalogue.

You mean like this? https://catagolue.appspot.com/textcensus/b3s23/C1

I forgot about those.

There should also be a page explaining the apgcode syntax.

You mean like this? https://catagolue.appspot.com/help/wechsler.txt

I had absolutely no idea that one existed!

[gameoflifeboy]

Actually, both subpages on https://catagolue.appspot.com/help/ are pretty useful.

I had absolutely no idea that one existed!

There used to be a link to the help page next to the statistics page. I don't know why it got unlinked.

[Apple Bottom]

You mean like this? https://catagolue.appspot.com/help/wechsler.txt

Speaking of that file, could you by any chance be persuaded to amend it to describe the format used for the codes of linear-growth patterns as well? I'm a little puzzled over those, but I'm not quite ready to delve into the apgnano code to try and work out how they're generated just yet.

[SuperJedi224]

Is there a help page about the symmetry codes?

[Apple Bottom]

Is there a help page about the symmetry codes?

There was a thread about them recently, here, with both textual explanations and pictures.

[thunk]

Is there a help page about the symmetry codes?

There was a thread about them recently, here, with both textual explanations and pictures.

Speaking of which, couldn't be 32*32 initial seeds used in the symmetric searches be overlapped slightly, with the overlapping parts XOR'd to produce a single value? The resulting 28*28 pattern should still be symmetric (if I'm thinking straight), and will fit in the starting box of apgnano.

EDIT: I just realised that there would be a problem with orthogonal or diagonal blank strips for the odd symmetries. Not sure how much of an issue that would actually be, or whether they could be filled in somehow.

[Scorbie]

EDIT: I just realised that there would be a problem with orthogonal or diagonal blank strips for the odd symmetries. Not sure how much of an issue that would actually be, or whether they could be filled in somehow.

Actually, that problem would have been in the original python script regardless of soup size if symmetries were implemented that way. Fortunately, they weren't. The python script fills in only the half/quarter/semi-quarter, with edges/diagonals/centers included, and decalcomanies them. So it wouldn't have that annoying odd-strip problem.

The thing I'm curious is: Would Catagolue accept 28x28 soups if we did make and census one?

[Scorbie]

EEK! I get errors when compiling apgnano!
The error message I got was a bunch of:

Code: Select all

life128.h:1370:183: error: unknown register name ‘xmm14’ in ‘asm’

I use 32-bit Linux, with Intel Atom CPU (which is listed on CPUs that support SSE2, SSE3 instructions)
Googling told me to check if g++ enables -mmmx or -msse2 by default and add the flags if not, so I did.
Fewer errors popped out. Before all 'xmm1~xmm14' caused errors but now it's 'xmm8~xmm14'.
What could be the problem and how could I solve it? If the solution is merely adding compiler flags then I think it would be wise to add the flags to the makefile in apgnano.git.

[calcyman]

I use 32-bit Linux

That's the problem; apgnano only works with x86-64 machines:

https://en.wikipedia.org/wiki/Streaming ... #Registers

The AMD64 extensions from AMD (originally called x86-64) added a further eight registers XMM8 through XMM15, and this extension is duplicated in the Intel 64 architecture.

I didn't realise people still used 32-bit machines. Sorry!

[Scorbie]

I use 32-bit Linux

That's the problem; apgnano only works with x86-64 machines.

The AMD64 extensions from AMD (originally called x86-64) added a further eight registers XMM8 through XMM15, and this extension is duplicated in the Intel 64 architecture.

I didn't realise people still used 32-bit machines. Sorry!

Didn't realize! I did see that apgnano only supports "x86-64, which is mostly the case except for really old hardware." And yes, I did try to install apgnano on a crappy old machine (my 5-yr-old netbook) but here I do see a lot of 32bit machines out there. (Even in my home there are two 32bit machines.) I thought you wanted to say x86/x86-64. Whoops.
Is there any obvious way to fix apgnano to work in 32bit machines? (Possibly at the cost of not using assembly?) Probably not, just hoping...

[Apple Bottom]

I didn't realise people still used 32-bit machines. Sorry!

Never underestimate the amount of legacy hardware (and software, for that matter) out in the wild - you'd be surprised what people still have running. And many folks got older machines they could dust off and repurpose as soup searchers given the opportunity.

Is there any obvious way to fix apgnano to work in 32bit machines? (Possibly at the cost of not using assembly?) Probably not, just hoping...

Just tossing out a random thought: does apgnano run several soups in parallel the way apgsearch 1.x did? If so, perhaps it could be made to only search half as many at the same time, only requiring half the number of registers, seeing as how the first 8 are available in 32-bit mode as well.

- Apple Bottom, who's always happy to add to someone else's workload

[Scorbie]

- Apple Bottom, who's always happy to add to someone else's workload

Laughing out loud

Just tossing out a random thought: does apgnano run several soups in parallel the way apgsearch 1.x did? If so, perhaps it could be made to only search half as many at the same time, only requiring half the number of registers, seeing as how the first 8 are available in 32-bit mode as well.

Wait a minute!!! If apgnano doesn't accept soups sized > 28x28 then it must mean it doesn't run soups in parallel, does it? And running soups in parallel made a 4x speed boost in apgsearch.py!

EDIT: @wildmyron Ahh, you're right. I was mistaken. Yes, apgsearch does the censusing part parallel, rather than the generating part, and yeah, I also agree that calcyman had optimized apgnano pretty well. I'm pretty sure he does the censusing parallel.

[wildmyron]

The python version doesn't (and has never) run the soups in parallel. It runs sqrtspp^2 soups serially until stabilisation and then performs the census in parallel on the ashes of all of the soups in one universe. I suspect this is primarily because the time to stabilisation is unknown and so it is difficult to determine how far apart to place soups to ensure there is no interaction between them. Once all the soups have turned to ash however, the maximum number of generations to stabilisation determines the required spacing between ashes in the census.

I haven't looked at apgnano, but in light of calcyman saying that the census is performed directly in C code (i.e. without the use of auxillary rules) I presume the census is not done in parallel. Even if it was possible to run the soups in parallel (in apgnano) I don't believe there would be any speed advantage because the Life128 algorithm is not hashtile based. Details are on the first page of this thread.

[Saka]

I recently uploaded the first soup with xs22_g9f033zdl913, but when I visit the page, for the still life, it says there are no soups stored in the catagolue. why isn't it "There is one soup stored in the catagolue"?
(https://catagolue.appspot.com/object/xs ... l913/b3s23)

[biggiemac]

I recently uploaded the first soup with xs22_g9f033zdl913, but when I visit the page, for the still life, it says there are no soups stored in the catagolue. why isn't it "There is one soup stored in the catagolue"?
(https://catagolue.appspot.com/object/xs ... l913/b3s23)

It lists one now. I think there is a 15 minute latency. Say you submit a haul at 15:11. At 15:15 that haul will enter limbo, where it awaits verification from some other instance. Almost always there will be verification before the next cron at 15:30. Then the objects will appear at 15:30.

[calcyman]

(No, apgnano doesn't run multiple soups in the same universe because there's no rule-switching overhead. The only reason to do parallel censusing in apgsearch v1.x is the time taken by Golly to switch between rules, which themselves were only necessary since manipulating lists of cells in Python is incredibly slow. In C++, these weird performance hacks are no longer effective.)

By the way, we've recently passed 2^45 objects in the Catagolue. #brimfulofhashes

One of the more recent objects was the thirteenth distinct p30 oscillator (found by @doctroid, earning a Monarchist badge in the process: https://catagolue.appspot.com/user/@doctroid ).

[Apple Bottom]

By the way, we've recently passed 2^45 objects in the Catagolue. #brimfulofhashes

And, now, 40 trillion objects. That's a barnful!

[Saka]

How fast does apgnano run on other people's computer? Is it just my computer being slow or what? People usually upload 20 million soups per haul, how long does processing 20 million soups take?

[thunk]

How fast does apgnano run on other people's computer? Is it just my computer being slow or what? People usually upload 20 million soups per haul, how long does processing 20 million soups take?

In my experience, running 10-million soup hauls on my two computers (desktop and laptop, both with core i7 processors), took 215 minutes per core for the former and 200 minutes for the latter. As stated in one of my posts above, this became inexplicably slightly slower with time (I have since stopped running soups, at least for now while I don't have access to the desktop.) At 8 cores in my desktop and 4 in my laptop, the total time to create 20 million soups was about 34 minutes.

[drc]

How fast does apgnano run on other people's computer? Is it just my computer being slow or what? People usually upload 20 million soups per haul, how long does processing 20 million soups take?

That's probably me. It takes usually 6 hours (slow computer)

[Saka]

Err...
Nevermind