Catagolue Discussion Thread

For general discussion about Conway's Game of Life.
User avatar
Apple Bottom
Posts: 1034
Joined: July 27th, 2015, 2:06 pm
Contact:

Re: Catagolue Discussion Thread

Post by Apple Bottom » October 14th, 2018, 5:05 am

calcyman wrote:No, I'm afraid they're irrevocably deleted.
Oh. Hmm, that's a bit of a bother as far as the statistics in the Catagolue article on the LifeWiki article are concerned. The C1 ones are OK, but the !C1 ones are aggregated across all higher symmetries, so I can't just use /objcount for those.

Looks like I may just have to remove the xp2 statistic. (It's probably not important exactly how many different period-2 oscillators were found in higher symmetries anyway.)

EDIT: or, alternatively, add a note saying it's not an exact figure. Caveat lector!
If you speak, your speech must be better than your silence would have been. — Arabian proverb

Catagolue: Apple Bottom • Life Wiki: Apple Bottom • Twitter: @_AppleBottom_

Proud member of the Pattern Raiders!

User avatar
muzik
Posts: 5614
Joined: January 28th, 2016, 2:47 pm
Location: Scotland

Re: Catagolue Discussion Thread

Post by muzik » November 5th, 2018, 7:25 am

Will any more rules be getting animated pattern icons in the near future? This one, for example, is the strobing equivalent of day and night, so if it would be possible for catagolue to somehow pick up on this, it could borrow animations from the real day and night.

User avatar
Ian07
Moderator
Posts: 891
Joined: September 22nd, 2018, 8:48 am
Location: New Jersey, US

Re: Catagolue Discussion Thread

Post by Ian07 » November 6th, 2018, 7:31 am

Adam, could you please delete the erroneous objects I submitted for Move/Morley and D&N real quick? Thanks.

User avatar
77topaz
Posts: 1496
Joined: January 12th, 2018, 9:19 pm

Re: Catagolue Discussion Thread

Post by 77topaz » November 6th, 2018, 8:00 am

Ian07 wrote:Adam, could you please delete the erroneous objects I submitted for Move/Morley and D&N real quick? Thanks.
I don't think there's any quick way for him to delete "erroneous objects"... otherwise we wouldn't have hundreds of censuses that contain them. There's even a few in b3s23 censuses, if I remember correctly.

User avatar
calcyman
Moderator
Posts: 2932
Joined: June 1st, 2009, 4:32 pm

Re: Catagolue Discussion Thread

Post by calcyman » November 6th, 2018, 8:31 am

Ian07 wrote:Adam, could you please delete the erroneous objects I submitted for Move/Morley and D&N real quick? Thanks.
I believe I already did -- I quarantined and backed out the offending hauls. Certainly there are no gliders here:

https://catagolue.appspot.com/census/b3678s34678/C1/xq4

or here:

https://catagolue.appspot.com/census/b368s245/C1/xq4

Oh gosh you've added four new hauls with gliders. Can you please stop searching until I realise what's happened (probably a bad regex on my part)? Thank you!
What do you do with ill crystallographers? Take them to the mono-clinic!

User avatar
Ian07
Moderator
Posts: 891
Joined: September 22nd, 2018, 8:48 am
Location: New Jersey, US

Re: Catagolue Discussion Thread

Post by Ian07 » November 6th, 2018, 4:34 pm

calcyman wrote:Oh gosh you've added four new hauls with gliders. Can you please stop searching until I realise what's happened (probably a bad regex on my part)? Thank you!
Sorry about that. I had a search running while I was at school and when I got back and heard there was a new update, I immediately submitted the hauls not knowing they contained the gliders.

User avatar
77topaz
Posts: 1496
Joined: January 12th, 2018, 9:19 pm

Re: Catagolue Discussion Thread

Post by 77topaz » November 6th, 2018, 5:07 pm

Oh okay, those erroneous objects were just coming from specific hauls from a specific bugged version. Some rules have been reporting erroneous objects in pretty much every version of apgsearch...

User avatar
calcyman
Moderator
Posts: 2932
Joined: June 1st, 2009, 4:32 pm

Re: Catagolue Discussion Thread

Post by calcyman » November 6th, 2018, 7:17 pm

Ian07 wrote:
calcyman wrote:Oh gosh you've added four new hauls with gliders. Can you please stop searching until I realise what's happened (probably a bad regex on my part)? Thank you!
Sorry about that. I had a search running while I was at school and when I got back and heard there was a new update, I immediately submitted the hauls not knowing they contained the gliders.
Thanks. I've now backed out each of those hauls individually from the census, and checked that it results in the number of gliders being restored to exactly zero.

Note to everyone: please update to v4.62-ll2.0.16 of apgsearch as soon as possible!
What do you do with ill crystallographers? Take them to the mono-clinic!

User avatar
muzik
Posts: 5614
Joined: January 28th, 2016, 2:47 pm
Location: Scotland

Re: Catagolue Discussion Thread

Post by muzik » November 20th, 2018, 9:05 am

Is it possible to request that certain censuses be merged together, provided the contained rules are equivalent?

User avatar
Apple Bottom
Posts: 1034
Joined: July 27th, 2015, 2:06 pm
Contact:

Re: Catagolue Discussion Thread

Post by Apple Bottom » November 24th, 2018, 4:53 am

While we're on the subject of requests, is this sort of thing really necessary? It's not even marked _Test.
If you speak, your speech must be better than your silence would have been. — Arabian proverb

Catagolue: Apple Bottom • Life Wiki: Apple Bottom • Twitter: @_AppleBottom_

Proud member of the Pattern Raiders!

User avatar
muzik
Posts: 5614
Joined: January 28th, 2016, 2:47 pm
Location: Scotland

Re: Catagolue Discussion Thread

Post by muzik » December 4th, 2018, 9:12 am

Would it at all make sense to add the ability to "report" objects which are incorrectly separated?

wwei23

Re: Catagolue Discussion Thread

Post by wwei23 » December 6th, 2018, 10:55 am

1x256X2, 1X2x256, D2_+1_Gutter, and the infamous 1x256X2+1, are listed under Unofficial symmetries in the symmetry list on rule pages, and as troll symmetries on object pages. They're supposed to be listed as troll symmetries on the rule pages.

User avatar
muzik
Posts: 5614
Joined: January 28th, 2016, 2:47 pm
Location: Scotland

Re: Catagolue Discussion Thread

Post by muzik » December 6th, 2018, 11:57 am


User avatar
calcyman
Moderator
Posts: 2932
Joined: June 1st, 2009, 4:32 pm

Re: Catagolue Discussion Thread

Post by calcyman » December 12th, 2018, 12:32 pm

Apple Bottom wrote:
calcyman wrote:It's not a bug; it's a feature.

[...]
Aha! That limit hadn't been on my mind anymore. Thanks for the explanation! :)

Is there a way to retrieve the culled objects, BTW? I.e., is there such a thing as (say) https://catagolue.appspot.com/textcensu ... D8_4/xp2/2 to get the second MB worth of objects, etc.?
Great news: I've now implemented gzip compression for large ( > 750 KB) b3s23 tabulations*. For instance, the xp2 tabulation in b3s23/D8_1 now occupies only 338438 bytes in storage (980063 bytes uncompressed).

* Frighteningly, this momentarily caused the xp2 tabulation in b3s23/D8_1 to display 0 objects on the census page, but that's because the Java code snippet I borrowed didn't remember to insert linebreaks in the decompressed string. After amending the code, everything works smoothly.

This means that b3s23 tabulations can now grow up to three times as large as before (and, in particular, the b3s23/C1 census won't start discarding objects until it reaches an object count of ~ 40 quadrillion -- approximately 150 times the current size of the census).

Now that we know that it works, I might start gzipping tabulations in other rules and lower the size threshold. This should lower memory and storage costs, theoretically, and therefore give a slight scalability boost to Catagolue.

EDIT: In retrospect, it looks like bzip2 would be even smaller than gzip (235664 bytes instead of 338438).
What do you do with ill crystallographers? Take them to the mono-clinic!

User avatar
Apple Bottom
Posts: 1034
Joined: July 27th, 2015, 2:06 pm
Contact:

Re: Catagolue Discussion Thread

Post by Apple Bottom » December 12th, 2018, 1:52 pm

calcyman wrote:Great news: I've now implemented gzip compression for large ( > 750 KB) b3s23 tabulations*. For instance, the xp2 tabulation in b3s23/D8_1 now occupies only 338438 bytes in storage (980063 bytes uncompressed). [...]

This means that b3s23 tabulations can now grow up to three times as large as before (and, in particular, the b3s23/C1 census won't start discarding objects until it reaches an object count of ~ 40 quadrillion -- approximately 150 times the current size of the census).
Marvellous!
calcyman wrote:EDIT: In retrospect, it looks like bzip2 would be even smaller than gzip (235664 bytes instead of 338438).
And then there's xz, as well as more exotic ones like zopfli and brotli... though with (de)compression also using resources, squeezing out a few more bytes may not be a good trade-off (right now).

Matt Mahoney's got an interesting comparison page here for various compressors, BTW.
If you speak, your speech must be better than your silence would have been. — Arabian proverb

Catagolue: Apple Bottom • Life Wiki: Apple Bottom • Twitter: @_AppleBottom_

Proud member of the Pattern Raiders!

User avatar
muzik
Posts: 5614
Joined: January 28th, 2016, 2:47 pm
Location: Scotland

Re: Catagolue Discussion Thread

Post by muzik » December 13th, 2018, 5:55 pm

Could a "moderator" class be added to Catagolue to help out with certain maintenance tasks? There's a few I have in mind:

- ability to propose names for currently unnamed Life objects, or request that existing names be changed if they're inconsistent with other names on the site,

- ability to delete or report spam comments or page-breaking comments, since there's still a few of those floating around,

- ability to flag badly-separated objects and incorrectly-classified objects in censuses, so they can be removed from the census or hidden form view,

- ability to flag bad hauls, such as that one that my computer posted with the adjustable spaceship rule, so that they can be backed out of the census,

- ability to flag certain rules with non-canonical rulestrings (i.e. rulestrings that the current version of apgsearch does not consider as valid) so that they can be merged into the canonical census or removed.

User avatar
Apple Bottom
Posts: 1034
Joined: July 27th, 2015, 2:06 pm
Contact:

Re: Catagolue Discussion Thread

Post by Apple Bottom » December 14th, 2018, 5:00 am

muzik wrote:Could a "moderator" class be added to Catagolue to help out with certain maintenance tasks? There's a few I have in mind:
You can already do all these things on the forums. The issue, I think, isn't so much reporting these things, it's making the changes. That -- as well as going through all the proposed changes to determine which have merit in the first place -- takes time, and time's limited. Put more succinctly, Calcyman does not scale; there's only one of him.

This is also why all these posts of "could X be done", "will we ever see Y", etc., are (generally) less than useful. Ideas (even good ones) are a dime a dozen; implementing them is more costly. As Edison famously observed, innovation is 1% inspiration, and 99% perspiration.

And although you can't hack on Catagolue, you can hack on e.g. apgluxe or Golly; so next time you've got a good idea, instead of suggesting that someone else take on the "99% perspiration", perhaps instead ask if a patch would be welcome, and if the response is positive, try implementing your idea yourself and send a pull request.

If you manage to implement your idea, great. And if you don't, perhaps you'll understand a bit better why all these aren't being eagerly jumped. (And if you're now thinking about replying "but I don't know Python / C++ / ...", keep in mind that nobody was born knowing anything about programming. Everyone had to learn it, so take it as your opportunity to educate yourself, rather than as an excuse to not learn anything new.)
If you speak, your speech must be better than your silence would have been. — Arabian proverb

Catagolue: Apple Bottom • Life Wiki: Apple Bottom • Twitter: @_AppleBottom_

Proud member of the Pattern Raiders!

User avatar
calcyman
Moderator
Posts: 2932
Joined: June 1st, 2009, 4:32 pm

Re: Catagolue Discussion Thread

Post by calcyman » December 20th, 2018, 2:07 pm

Apple Bottom wrote:
calcyman wrote:EDIT: In retrospect, it looks like bzip2 would be even smaller than gzip (235664 bytes instead of 338438).
And then there's xz, as well as more exotic ones like zopfli and brotli... though with (de)compression also using resources, squeezing out a few more bytes may not be a good trade-off (right now).

Matt Mahoney's got an interesting comparison page here for various compressors, BTW.
Thanks! This is very interesting in general; I'm not sure how well this will apply to Catagolue's tabulations, though, because the data are of very different formats (Wikipedia is natural language, whereas Catagolue's tabulations are lists of apgcodes and frequency counts).

It looks like bzip2 is consistently about 30% smaller than gzip, but also considerably slower. Consequently, Catagolue now operates the following approach:
  • Always limit the uncompressed length of the tabulation to 980000 * R bytes, where R is the last estimate of the compression ratio. Beyond that, the least common objects in the tabulation are discarded.
  • Ordinarily use no compression (R = 1.0).
  • When the tabulation first exceeds 300000 bytes, calculate the compression ratio R by gzipping it. The assumption is that the true ratio will remain close to R even as the tabulation continues to grow.
  • When the tabulation first exceeds 900000 * R bytes (where R was determined above), calculate R_g and R_b (the gzip and bzip2 compression ratios) and set R to be the greater of the two. Almost certainly bzip2 will win by a clear margin here, but I don't want to run the risk of increasing the compressed size by switching to bzip2.
This balances computation time, compressed size, and accuracy in estimating compressed size. The slower bzip2 algorithm will only be invoked when the estimated size of the gzipped tabulation exceeds 900 kB.

For b3s23/C1, the upshot is that when we reach ~ 40 * 10^15 objects (!), the largest tabulation will switch from gzip to bzip2, allowing the census to comfortably grow to 85 * 10^15 objects (as opposed to 50 * 10^15 objects under gzip, or 3 * 10^15 objects without compression) before shedding obscure still-lifes.

@AppleBottom -- do you have a comprehensive list of all of the xp2 oscillators that 'went missing' from the D8_4 and D8_1 tabulations? I guess now is an appropriate time to reintroduce them.

Viva Catagolue!
What do you do with ill crystallographers? Take them to the mono-clinic!

User avatar
Apple Bottom
Posts: 1034
Joined: July 27th, 2015, 2:06 pm
Contact:

Re: Catagolue Discussion Thread

Post by Apple Bottom » December 20th, 2018, 5:00 pm

calcyman wrote:For b3s23/C1, the upshot is that when we reach ~ 40 * 10^15 objects (!), the largest tabulation will switch from gzip to bzip2, allowing the census to comfortably grow to 85 * 10^15 objects (as opposed to 50 * 10^15 objects under gzip, or 3 * 10^15 objects without compression) before shedding obscure still-lifes.

@AppleBottom -- do you have a comprehensive list of all of the xp2 oscillators that 'went missing' from the D8_4 and D8_1 tabulations? I guess now is an appropriate time to reintroduce them.

Viva Catagolue!
Vivat!

Sounds like a very sensible trade-off between avoiding increased resource usage for (de)compression and allowing room for future growth.

As for the missing objects... yes, I just went through all the old files I have and generated what I believe are[1] exhaustive lists of objects missing from B3/S23/D8_1 and B3/S23/D8_4, using current Catagolue data (i.e., just downloaded a moment ago). They're fairly large, so I'm attaching them to this post as an archive file.

1. Famous last words...
Attachments
missing.tar.gz
(238.52 KiB) Downloaded 276 times
If you speak, your speech must be better than your silence would have been. — Arabian proverb

Catagolue: Apple Bottom • Life Wiki: Apple Bottom • Twitter: @_AppleBottom_

Proud member of the Pattern Raiders!

User avatar
calcyman
Moderator
Posts: 2932
Joined: June 1st, 2009, 4:32 pm

Re: Catagolue Discussion Thread

Post by calcyman » December 20th, 2018, 6:05 pm

Apple Bottom wrote:As for the missing objects... yes, I just went through all the old files I have and generated what I believe are[1] exhaustive lists of objects missing from B3/S23/D8_1 and B3/S23/D8_4, using current Catagolue data (i.e., just downloaded a moment ago). They're fairly large, so I'm attaching them to this post as an archive file.
Thanks! I believe the censuses should now be correct thanks to these emergency hauls:

https://catagolue.appspot.com/haul/b3s2 ... ommitted=6
https://catagolue.appspot.com/haul/b3s2 ... ommitted=6

I've checked using the command-line tool comm that each new textcensus is exactly the union of the corresponding old one and the 'missing.txt' file.
What do you do with ill crystallographers? Take them to the mono-clinic!

User avatar
muzik
Posts: 5614
Joined: January 28th, 2016, 2:47 pm
Location: Scotland

Re: Catagolue Discussion Thread

Post by muzik » December 22nd, 2018, 7:45 am

Can the boxes containing sample soups be made collapsible (and closed by default, maybe except for the official symmetries)? They tend to take up quite a lot of space, which isn't exactly a good thing.

User avatar
Lewis
Posts: 337
Joined: March 17th, 2009, 5:26 pm
Location: UK
Contact:

Re: Catagolue Discussion Thread

Post by Lewis » December 25th, 2018, 7:14 am

Some of the oscillator list pages for 2x2 symmetries are not sorting the objects by number of occurrences, e.g. https://catagolue.appspot.com/census/b36s125/D8_4/xp4
https://catagolue.appspot.com/census/b36s125/D8_4/xp10

User avatar
calcyman
Moderator
Posts: 2932
Joined: June 1st, 2009, 4:32 pm

Re: Catagolue Discussion Thread

Post by calcyman » December 26th, 2018, 8:08 am

Lewis wrote:Some of the oscillator list pages for 2x2 symmetries are not sorting the objects by number of occurrences, e.g. https://catagolue.appspot.com/census/b36s125/D8_4/xp4
https://catagolue.appspot.com/census/b36s125/D8_4/xp10
If you commit a new haul to that census, it should sort itself out. (That census is ancient -- last updated May 2015 -- so probably before I ensured that tabulations are internally sorted.)
What do you do with ill crystallographers? Take them to the mono-clinic!

wwei23

Re: Catagolue Discussion Thread

Post by wwei23 » December 26th, 2018, 7:45 pm

Catagolue's robots.txt doesn't let The Wayback machine create an archive of Catagolue, yet there are still archives of Catagolue in the Wayback machine. How is this possible?

User avatar
muzik
Posts: 5614
Joined: January 28th, 2016, 2:47 pm
Location: Scotland

Re: Catagolue Discussion Thread

Post by muzik » December 27th, 2018, 6:40 pm

Is it absolutely neccessary that lists of oscillators and the like use animated previews? It tends to lag the page out a lot, especially with large amounts of higher periods.

Post Reply