Searchable database of comics?

[snipertom]

Just wondering, has anyone thought of making a searchable database of each comic with the setting(s), characters, props, etc?

EDIT EDIT: THIS THING NOW EXISTS!!!!! It's at www.louisxiv.co.uk/gunnerkrigg/ and is totally working and RADICAL DUDEZ

EDIT: For those who want to get started, there is a github here just ask me or louisxiv for access, or people can post transcribed comics on this thread (best to copy from a text editor into a post between code tags). Directions as follows:

Tagging
- What to tag? Anything relevant that can be seen in the page that people might want to search for
- The top priority is to tag all the characters, settings and important props; other things can be added later if you're not sure
- It's best to use tags that already exist (check alltags.txt) if possible. If you need to add a new tag, add it to alltags.txt at the same time with a description; use tags such as "GoodHope" rather than "GoodHopeHospital" as tags can be described
- At the moment, just tag characters who are seen (in the main action, flashbacks, memories), rather than mentioned (this may change)
- Important props/items such as the blinker stone need to be tagged
- Recurring characters should be tagged by name or a specific description if possible; generic characters should be referred to as "teacher" or "students" or "LaserCow"
- If you can tag things like alchemical symbols that would be very useful. If you are not sure what a symbol is, tag it as "symbol" and make a new post on the forum and ask- there are a lot of knowledgeable people out there!
- The forum, tvtropes and the Gunnerkrigg Wiki (gunnerkrigg.wikia.com) are invaluable for information on who and what exists in the comic
- At the moment we are not tagging the cover pages however this might change
- Treatises are obviously full of tag-material! Have a go and we'll all contribute further tags as necessary

Index of pages - Index.txt
- The comics are in reverse order
- Each comic is on a separate line, starting with the page number (look at the URL of the comic and find the number at the end)
- Each item is separated by tab-spacing
- The tags are in alphabetical order and include relevant locations, characters, symbols and items
- Page titles are preceded by "#"
- The order is [page number] [tag] [tag] ... [comment] [page title]
- Covers are simply: [page number] Cover #<hr>[Title of cover]<hr>
- Bonus pages are: [page number] Bonus [tags] #<b>Bonus Page:</b> [What is in the bonus page]<hr>

eg
16 Bonus Blackboard Chester Foley Queslett Tea Thornhill #<hr>Bonus Page: Houses at Gunnerkrigg Court<hr>
15 Annie Bridge Gillitie Teacher #Sorry sir. I got lost.
14 Bridge Gillitie Robot Shadow Shadowmen #Here you go little buddie!
13 Bridge Robot Shadow TicToc #Oh look! A birdie!
12 Annie Bridge Robot Shadow #Never fear little guy. I have this under control!
11 Annie Bridge Robot Shadow #Once darkness fell, Shadow 2, Robot and I got ready at the foot of the bridge
10 Annie Bridge Robot Shadow #Together the three of us returned to the bridge
9 Annie Robot Shadow #Luckily, piecing the contraption together was relatively easy
8 Annie Shadow Library #Snatching a nearby box of spare parts
7 Annie Shadow Library #I hadn’t the first idea how to build a robot
6 Annan Annie Shadow #I must construct a robotic walking device
5 Annan Annie Bridge Gillitie Shadow #Gunnerkrigg Court is a boarding school
4 Annie Shadow #Where are you trying to go?
3 Annie Shadow #At times I would ssee this creature jump from shadow to shadow
2 Annie Shadow #Gunnerkrigg Court does not look much like a school at all
1 Annie Cover #<hr>Chapter 1: The Shadow and The Robot<hr>

Tags & descriptions - Alltags.txt
- The tags are in alphabetical order
- Each tag is on a separate line
- The tag and its description are separated by tab-spacing: [tag] [description]

eg
AnimalCells the secret Court large animal holding cells
AnimalLab a Court research lab
Anja Anja Donlan, Kat's mother
Annan Annan Waters, a wide river in a deep gorge
Annie Antimony Carver, our protagonist
AntimonySymbol The alchemical symbol for antimony; associated with Annie and Surma
Basil Minotaur
BeckyGround A student
BismuthSymbol The alchemical symbol for bismuth; associated with the Court
Blush happy/embarrassed
Bob Bob Sutton, a gardener/maintainer of artificial habitats and keeper of Young's Park with his wife, Marcia
Bobbie A robot at the AnimalLab
Bonus Extra pages at the end of the chapter
Bridge over the Annan Water to Gillitie Wood
Brinnie Sultry childhood friend of Surma, Anja, Donny, Eggers & co.
Chickcharney see Hardwick & Little’s Bestiary, p65
CherryTree Comes as advertised
Chester One of the other school Houses
Circus creepy and filled with clowns
Classroom Where they have classes
Clowns creepy, no-one likes them

Starting with GitHub
- First you need one of us (probably me) to give you access to the repo. Please send me a PM or reply on this post if you want access.
- You can either edit the files directly on the website and commit changes that way or:
- You can download a program for win/mac/linux
- Install it and run it
- Sign in and click on your username below "github" on the far left of the program
- Click on the repository in the middle that you want (snipergirl/gunnerkriggg) and click 'clone'; that will give you a copy on your computer.
- Each time before editing, in the main part of the program can click "refresh" at the top middle, click on the repo and click "sync" up the top.
- Then edit the files yourself
- When you are ready to submit your changes, save the file
- Go into your github program, which will show that you have unsaved changes in whichever file(s)
- Put in a commit message about what you have done in the right hand area and click "commit"
- Then click "sync" and it should upload.
- This is also a tutorial about Git in general: try.github.io/levels/1/challenges/1

Current work allocation:
- louisxiv going backwards from the end
- snipergirl doing even chapters from the start
- Morpheus doing odd chapters from the start
- spritznar doing comics from 844
- philman doing comics from 503

[louisxiv]

I’ve done something like that for Misfile - an index listing each comic page and the significant characters appearing therein, with a simple mechanism allowing selection of all the pages a character, or tagged object, appears in.

I was just thinking the other day that it would be easy to do the same for Gunnerkrigg – apart from the time taken to actually review and enter details of all the pages.

Misfile: www.misfile.com/
Index at: www.louisxiv.co.uk/misfile/

[snipertom]

yes! like that! i guess one could make it such that various people could sign up and enter data whenever they felt like it, to lighten the load?

[louisxiv]

Not possible to open it on my space I'm afraid.

As the basics are a text file that I run through a script to generate the body of the page, it would be possible for other people to make the basic input file in chunks (per book or whatever) and send it in to be compiled – not exactly yer aksual modern inneractiv web experience, but good enough to get over a set-up hump.

Just a question of everyone using the same tag set - is she Ann, Annie, Antimony or Carver…

For Misfile:
2013-05-23 (tab) Ash (tab) Ivan (tab) OldRoad (tab) #commentary or title here


(which is PageID, a couple of character tags, a location tag and a comment)

I could host it, no problem, but I'm happy to talk someone enthusiastic though the details how it all works so they can do their own thing. Needs a bit of web hosting space and some simple-ish editing skills, is all.

[snipertom]

May 26, 2013 14:04:11 GMT louisxiv said:

Not possible to open it on my space I'm afraid.

As the basics are a text file that I run through a script to generate the body of the page, it would be possible for other people to make the basic input file in chunks (per book or whatever) and send it in to be compiled – not exactly yer aksual modern inneractiv web experience, but good enough to get over a set-up hump.

Just a question of everyone using the same tag set - is she Ann, Annie, Antimony or Carver…

For Misfile:
2013-05-23 (tab) Ash (tab) Ivan (tab) OldRoad (tab) #commentary or title here


(which is PageID, a couple of character tags, a location tag and a comment)

I could host it, no problem, but I'm happy to talk someone enthusiastic though the details how it all works so they can do their own thing. Needs a bit of web hosting space and some simple-ish editing skills, is all.

I could also host it if needs be, and I'm keen to learn! Outputting the data seems straightforward, the 2 things that I'm a bit stuck on are the database structure and how to make the input form accept multiple characters as well as new characters (yes, it's stupid, I know!)

EDIT: I suppose you could have a lookup table with aliases for each character as well?

[louisxiv]

What I've done is just a flat web page with a slightly smart css/js selection mechanism to hide 'unwanted' entries. No fuss, no mess, no SQL...

Take a look at my page source

[louisxiv]

You do need a wee css stylesheet for each individual tagged item for the selector to work - it is all done by mirrors and stylesheet swapping.

[snipertom]

ahh, i see how you did it!

[louisxiv]

Excellent!

[louisxiv]

www.louisxiv.co.uk/gunnerkrigg/

Ok, so I got carried away: I had intended just to run through half a dozen pages as a demo, but that's the first 14 pages and Chapter 42. It needs several hundred more pages added before it becomes useful though.

index.txt is the source file and
gunpy.txt is gun.py, a python script to convert index.txt to html to include/paste into the main page.

The script links (stylesheet.txt) and the tag index (tags.txt) are both generated by a series of greps from index.txt. I've got those wrapped up in a BBEdit textfactory if you happen to use BBedit... The comments/description on the tags are hand-edited.

The stylesheets for the switching trick are also hand edited, still, despite my intention to include it all in the python script - some day.

[GK Sierra]

May 26, 2013 19:34:49 GMT louisxiv said:

www.louisxiv.co.uk/gunnerkrigg/

Ok, so I got carried away: I had intended just to run through half a dozen pages as a demo, but that's the first 14 pages and Chapter 42. It needs several hundred more pages added before it becomes useful though.

index.txt is the source file and
gunpy.txt is gun.py, a python script to convert index.txt to html to include/paste into the main page.

The script links (stylesheet.txt) and the tag index (tags.txt) are both generated by a series of greps from index.txt. I've got those wrapped up in a BBEdit textfactory if you happen to use BBedit... The comments/description on the tags are hand-edited.

The stylesheets for the switching trick are also hand edited, still, despite my intention to include it all in the python script - some day.

Oh wow, that is awesome!

Bookmarked.

[snipertom]

very awesome indeed! how's best to update it with more comucks?

[spritznar]

May 26, 2013 19:34:49 GMT louisxiv said:

www.louisxiv.co.uk/gunnerkrigg/

Ok, so I got carried away: I had intended just to run through half a dozen pages as a demo, but that's the first 14 pages and Chapter 42. It needs several hundred more pages added before it becomes useful though.

index.txt is the source file and
gunpy.txt is gun.py, a python script to convert index.txt to html to include/paste into the main page.

The script links (stylesheet.txt) and the tag index (tags.txt) are both generated by a series of greps from index.txt. I've got those wrapped up in a BBEdit textfactory if you happen to use BBedit... The comments/description on the tags are hand-edited.

The stylesheets for the switching trick are also hand edited, still, despite my intention to include it all in the python script - some day.

this project sounds awesome and i feel like i should help,
but i do not have any idea what you just said.
(it might be more useful if i don't help)

[GK Sierra]

May 27, 2013 5:25:45 GMT spritznar said:

May 26, 2013 19:34:49 GMT louisxiv said:

www.louisxiv.co.uk/gunnerkrigg/

Ok, so I got carried away: I had intended just to run through half a dozen pages as a demo, but that's the first 14 pages and Chapter 42. It needs several hundred more pages added before it becomes useful though.

index.txt is the source file and
gunpy.txt is gun.py, a python script to convert index.txt to html to include/paste into the main page.

The script links (stylesheet.txt) and the tag index (tags.txt) are both generated by a series of greps from index.txt. I've got those wrapped up in a BBEdit textfactory if you happen to use BBedit... The comments/description on the tags are hand-edited.

The stylesheets for the switching trick are also hand edited, still, despite my intention to include it all in the python script - some day.

this project sounds awesome and i feel like i should help,
but i do not have any idea what you just said.
(it might be more useful if i don't help)

Don't worry, I have clipboards and hard hats for both of us so we can stand around looking useful.

[Morpheus]

Pass on the hats and stuff. I like this project.

[snipertom]

here is chapter 1,2 and 42!

EDIT: drat that doesn't seem to be working, will post it on github instead- here is it!

EDIT: added chapter 3

Attachments:

[philman]

Looks very nice, and would be great if more people could contribute.

So is the only editing required to add more pages editing the index file, as snipergirl has? As those additions don't seem to have been added to the database yet. Or does the owner have to manually enter them?

[snipertom]

It needs to be parsed by gun.py first, and louisxiv's site css files & html also need to be adjusted for the new items/tags, but in the meantime, editing the index.txt I think is the biggest task?

On the github there is now a file with all the tags so far called "alltags.txt" that should be able to be parsed with a script to create stylesheet.txt and tags.txt.

Basically, if people could update both index.txt and alltags.txt at the same time it would be great!

[louisxiv]

I spent a few minutes at lunchtime working out how to improve gun.py - the sheer sense of embarrassment at explaining the clunky lashup of the present system is, is ... stimulating.

<nerd>

I was thinking of adding tag descriptions to the main index.txt as embedded lines of

@ (tab) tag (tab) description

which would be easy to pull out and process by a putative gun2.py

</nerd>

I'll d/l snipergirl's updated index.txt (ta!) and update the site in a minute or 10: coffee calls...

[louisxiv]

O-kay, my GitHub-fu is weak, I confess - how do I d/l index.txt as a clean utf-8 text file from there...

[snipertom]

if you click on index.txt you can copy the text in it? alternatively, you can star the project, install the github program and ask it to download the repo for you. otherwise I can email you a copy?

This is friggin sweet by the way! Just checked out the updated site! It's so fast

[louisxiv]

Character index updated.

Took a bit of tidying - somewhere in its journey to GitHub the utf-8 index.txt got stomped into something lossy so all the smart quotes and ellipses were multi character smush. I've generally educated dumb quotes in the new bits of index.txt too, though I forgot to fix them in bits of the tags descriptions as I brought them in.

There's a clean index.txt in the same place as before, but I'll see about getting onto GitHub my self so I can fix it directly.

Thanks muchly to snipergirl.

[snipertom]

Clean index.txt has been uploaded to github. For some reason (I blame my internets or browser for doing something) when I downloaded the original index.txt there was that problem with the multicharacter weirdness (which I assumed was some sort of encoding thing that might resolve once it was on the intarweb). Sorry, i should have fixed that up!

[louisxiv]

May 27, 2013 19:33:34 GMT snipertom said:

This is friggin sweet by the way! Just checked out the updated site! It's so fast

Heh. Try the Misfile page on a not-current-generation mobile phone or tablet to see where that is going to go. Works fine on a desktop though, still.

Misfile is somewhere past 2300 pages, though I expect it'll have fewer tags in the end. Gunnerkrigg is a very "target rich environment" there if you start tagging the symbolism...

[snipertom]

Oh yeah definitely. not to mention the things that you only notice much later- like Jones' footprints in the paintings in the library and the alchemical symbols

It's easy to tag a lot of irrelevant shit, I hope that I haven't done it excessively!

[snipertom]

louisxiv added you to my repository so you should be able to upload & update stuff on there.
also can you add me as a collaborator on yours so I can merge it? github.com/louisxiv/gunnerkrigg/settings/collaboration


Anyone else want access?

[louisxiv]

May 28, 2013 9:54:03 GMT snipertom said:

louisxiv added you to my repository so you should be able to upload & update stuff on there.
also can you add me as a collaborator on yours so I can merge it? github.com/louisxiv/gunnerkrigg/settings/collaboration

Thanks - I realised I needed to do something to add you but couldn't see where in the depths of the night. Done, I hope.

[louisxiv]

If anyone with clipboard, helmet and nothing useful to do (!) wants to contribute, but thinks signing up to GitHub is a bit OTT, just post stuff here, maybe?

The essential stuff is the page number and a bunch of one word entries ("tags") for interesting things on that page - which characters and the like. Sticking it in a [ code ] [ /code ] block would be helpful


67 Annie Court   Mort #Thanks a bunch, Annie
66 Circus Clowns Paz


The following # and a page title/summary isn't essential - it can be added later, the heavy lifting is in getting the basic tagging done.

It might be worth a Cool Hand Luke shout-out post:

"Tagging Chapter 19 here, Boss",
or
"Boss: Tagging pages 800-820 here"

...so you don't trip over work someone else has started?

Keep an eye on the existing tags maybe, but don't get yer panties ina bunch over consistency. Later passes can to sort out glitches and duplicate tags.

If I see such a post I can cut'n'paste into the master index.txt with any necessary tidy-up.

[louisxiv]

May 28, 2013 7:26:48 GMT snipertom said:

Oh yeah definitely. not to mention the things that you only notice much later- like Jones' footprints in the paintings in the library and the alchemical symbols

It's easy to tag a lot of irrelevant shit, I hope that I haven't done it excessively!

From doing Misfile, I'd suggest

- It is probably better to over-tag than under, as we can take the irrelevant shit out later - it is a PITA trawling though 1200+ pages when you decide you want to add a new tag type you didn't use before, compared with a search and delete for whatever turns out to be irrelevant.

- not to list the Court itself as a tag, just because it is the default location for most of the action.

- Keep tags as concise as possible 'cause there's always the tag description for details: for instance GoodHope rather than GoodHopeHospital.

- Keep item tags as general as possible - you're more likely to pick up interesting coincidences / significant clues. F'rinstance JonesFootprintPictures might be better as Footprint and Pictures I suspect.

N.B. though: this classification business can become the sort of hairsplitting that entire academic careers are based upon...

[snipertom]

Ok, I've merged the repos and added chapter 5.

Agreed with louisxiv, if github isn't your thing (it can be quite the pain in the arse it seems at times!) then please post stuff here.

In fact, that might be even easier!

(108 down, just 1095 to go!)