Keyboard database

[Daniel Beardsmore]

OK, so let's suppose there was to be a proper community-driven keyboard database (make, family, model, switches used, dates found etc) that can be mined to provide product history for keyboards and switches.

We already have a unified login between MediaWiki and phpBB, which is one reason for having such a database directly on the Deskthority website — it should be possible to share phpBB's login system.

What should such a site be called?

Should it reside within Deskthority, or should it have its own domain and hosting and its own dedicated logins? If it were to live at Deskthority, should it be a subdirectory (like the wiki) or a subdomain?


Secondly, how would such a system be assembled? Consider that a single new entry may require the data entry of the brand (and its underlying company), the OEM, the switch manufacturer, the keyboard family or series, the switch family or series, multiple switch models and finally the details of the keyboard itself. Whatever UI it has must be able to cope with this level of concurrent entry for a while. No doubt there will be AJAX involved in this, and some kind of wizard engine.


It's got to be exceptionally user friendly, so that it's suitable for anyone to use.

[snuci]

In my opinion, It would be best if it was within Deskthority with same login credentials but I am of two minds as it should go beyond Deskthority to be effective. Let me explain.

This KeyboardDB site should be worthwhile for people to go to and use so my suggestion would be to make this a "what keyboards do I have" type user driven site so that one can keep their keyboard inventory, yet provide the valuable information for various models/keyswitches/dates, etc. It could give you a quick inventory and also show others what keyboards people have (if they want to make it public). It would help if someone is looking for a particular type of keyboard or wants some help from someone who has the same one.

Just throwing ideas out there but if it were useful as a personal tool, the information gathered in populating one's personal keyboard inventory would provide valuable information.

[Daniel Beardsmore]

In order to be able to mine the data, "me too" would be have to be prohibited. If someone had already entered a Dell AT102W, I couldn't declare that I have one in my collection by selecting "I have one of these" — I'd be required to document mine in detail. I can't see anyone wanting to document their collection unless detailed data entry were not a requirement. For example, not just date, but also the fact that many keyboard types have multiple switch types including occasionally switches from different manufacturers (chiefly Forward and Xiang Min switches together in both Matias and CVT keyboards).

A person may decide to enter some of their more interesting keyboards, but it would stop there unless the data entry requirements were very lax.

With that said, it would be difficult to enforce any data entry, as there are always unknowns: keyboards with no brand, no known OEM, totally unidentifiable switches (such as those wee switches in that alleged old Tai-Hao keyboard), and keyboards with nothing that is recognisable as a model number or part number.

As such, it would be governed more by a moral code than any technical enforcement, and that's not going to work.

[Wodan]

Oh what a cute idea!

I have actually been meaning to build something like this for my own keyboards, especially trying to keep track of the details of each build I made.

But this is even better ... for a name, why not go with KBDB <= KeyBoard DataBase
And I am a big fan of subdomains, wish the wiki had it's own subdomain!
Much easier to remember than something /kdsfjasdkfj

Would love to hack in all my recycler finds and random puchases just to fill the base a little with empirical data.

Should we discuss DB design?

I realized when starting some custom-keyboard articles in the wiki that the infobox for keyboards didn't quite fit the requirements for custom kits since there are other interesting factors that play a role there. Are we going with a small amount of essential parameters or shall we go all-in with dozens of potential parameters, only 15-20% of which will usually be populared?

This could also be an amazin tool when browsing ebay auctions and searching for model numbers.

PLEASE DO IT!!!

[Scarpia]

I may be able to assist with the DB and engine development, as I happen to be working on a side project which would be perfect for this. On the other hand, relying on a codebase created and maintained by just one person is rarely a good idea, not to mention the fact that this platform may not be ready for another few months.

Either way, it would be wise to consider a few things:

1) Searching / filtering should be a core concern, meaning it should use a proper search index like ElasticSearch or Solr.

2) Ideally it should come with at least some moderation tools, allowing Beardsmore and others to identify and fix errors. E.g. merging duplicate entries, as one person may add an entry for 'IBM Bigfoot' and another might add an entry for "IBM Model F 1397950" - and a third person might add "IMB Bigfoot"

3) This: "I can't see anyone wanting to document their collection unless detailed data entry were not a requirement." I for one would definitely not take the time to document my collection in detail unless I could do some form of "me too" copying of existing content. I would happily add details about the few uncommon boards I own, while I have no intention of being the 40th person to document another SGI Granite or AEK2.

The question is, what are we trying to achieve? To document all the individual keyboards in existence and their provenance (treating keyboards like race horses or famous diamonds)? Or to crowdsource the database for efficiency and better coverage, sharing the workload among the community rather than leaving all the work to Beardsmore and XMIT?

[Findecanor]

I see lots of potential for integrating such a keyboard database with the Wiki.
First we could get lists of valid entries (switch, etc.) from categories in the Wiki so that we wouldn't have to enter those manually.

But the killer feature could be if the Wiki could use the database for finding data to display in the Wiki about a keyboard model.
One example be the infobox for a keyboard: it could get data such as switches and production years from the database. Of course, the clauses in the infobox would need to support manual exceptions to cater for info that has been gathered some other way.
Another example would be an automatic table of known keyboard variations. Whenever someone enters a specific variation into the database, that would make that variation appear on the Wiki.

[marijan]

The question is, what are we trying to achieve?

I think a database is an excellent idea. Our current wiki is a great resource, but sometimes one just can't find the exact model since an entry doesn't exist or is buried somewhere in the old threads.

Here is what I would consider a great finished thing:
1) Searchable database where one can quickly find the exact keyboard and get all spec details for it (pictures included).
Each database item will have it's own (autogenerated?) web page listing all known details, including photos or links to photos. Basically, one should be able to get a static URL pointing to the exact keyboard model, similar how we can currently point to a wiki page.

2) Ability to create a personal inventory list, using the items found through feature 1).
If I find out that I have the exact same keyboard already described and documented, I would like to be able to add it to my personal inventory list. Also, I would like the option (need this option, actually) to modify some non-essential pieces of information, unique to my concrete keyboard (e.g. production date). This info can then be aggregated and shown on the main info page for this keyboard (e.g. number of known keyboards, known production date range, etc.)

3) Ability to add new database entries
If the exact same keyboard does not exist in the database, one should be able to create a new entry. One should not be forced to do this from scratch as we always want to avoid data duplication. Instead, one could simply select appropriate parts from the existing list (e.g. for AEK II with salmon alps, one could select the closest model he can find, let's say AEK II with white alps, and then edit the switch type). Or one can build from ground up, selecting the appropriate item from each category (select switch type, protocol, cable, case, layout, etc.).
If a certain part does not exist (e.g. no one has made an entry for salmon alps yet), then one is either forced to create an entry for it or cancel the whole thing until someone else creates an entry.

4) The whole thing should be moderated to some extent, so as to avoid intentional misinformation (e.g. creating entries for AEK with MX blue switches). Since this is a community project, we can have different levels of account access. The elders who can post immediately as they can be implicitly trusted, maybe a middle tier for those who can create new items (e.g. new switch type) but must accompany it with photos and/or other proof (this should be a golden rule for everyone as we really want to include as much knowledge as possible). And a novice tier who will need their posts approved before publishing, until their trust level increases and they get promoted.

5) Of course, existing entries can and should be expanded with new knowledge as it becomes available (you found some new info on Siemens, or on Tiptronic model 127, or have nice photos of some swithc? Add it to the database). As mentioned, novice entries to be subjected to approval process.

All of the above relies on database already having a certain amount of items. This means that for some time it will not be tremendously useful for general audience until we transfer the existing knowledge from wiki into the database. However, this does not have to be such a hard thing to do and I'm sure that a lot of us, novice users, would gladly volunteer. I may not know a lot about mechanical keyboards but I know how to copy/paste and organise the existing data into a new format.

[snuci]

Forgive my vintage computer collector examples but...

I couldn't find this yesterday but my example site for the "personal inventory" is something like this: http://www.old-computers.com/club/colle ... efault.asp It's not the best of examples as it looks somewhat broken but you get the general idea.

This is merely a sample with the focus being on the collector but in our case, the focus should be on the keyboard/key switch. With the appropriate information entered by collector, you can get features like "Trade List" or similar for those who may want to trade. These are merely useful (if you so choose) features that benefit the keyboard enthusiast that will entice them to post their keyboard collection. This could go beyond DT. The key for collecting information is the metrics and information that will result.

I would expect that users would list even the most common keyboards in their inventory so that we can build up any key switch variations and more importantly, dates. Switches can then be correlated back to dates so that given a big enough sample size, you might be able to figure out rough timelines for key switch availability. The data for mining would be great.

As mentioned, we should put in duplicates but as suggested, make it easy to select most information from pre-populated drop-down boxes with entry of new types as they are entered. A reasonable example of tracking variations and serial numbers is the Commodore 64 registry. http://c64preservation.com/registry This builds up a timeline for when certain revisions were released. I, myself, have some of my Commodore 64's in this.

Anyway, this could be very good for users and the information gathered can help the community as well.

[Daniel Beardsmore]

The question is, what are we trying to achieve?

Currently the date ranges for Alps switches on the wiki are based on MouseFan's chart, here:

http://mousefan.telcontar.net/alpsk.htm#chronology

There is disagreement over this data, and I am inclined to believe that MouseFan is incorrect. Without having any raw data, I wasn't able to prove it one way or the other, so my original goal was to create a database to store this raw data, which you can see here for SKCM series:

http://telcontar.net/KBK/Keycombo/switchseries.php?id=7

That project is essentially dead, but I've not given up on the idea. However, if we're going to do this, then it seems sensible to extend it to cover all reasonable goals, so long as we all agree on what is reasonable! So far I'm seeing good ideas. While I remain anti-club, I do think that it may be a nice idea to restrict some of the features to club members (such as cataloguing your personal collection) (which suits me as I have no plans to make use of that feature ;-)

This is the existing schema:



It's not nice, as it's a yucky data model, but it could be improved — the enums should go, for example, as we need the extra flexibility.

Features such as review and moderation were part of the original plan, as are permanent links — you can get a pretty good idea from what I've already made, but it's the data entry UI that's the main issue.

[codemonkeymike]

I would recommend using some domain specific attributes table. Like an "external_resource" table which could have all the citation information on the table. As well as I think you should have the enums be defined by the front-end not in the database especially if one of the enum values is "other". Personally I don't like the schema at all, it is very normalized and I am currently working on refactoring a database with a schema just like this.

[Daniel Beardsmore]

I am going to assume you meant "not very normalized".

I already said that the enums are wrong. The reference/2/3 isn't a schema mistake — it's a UI limitation. Where it's mandatory to have a one to many relationship (in particular switch types) this is catered for, but for references it was easier to put in three boxes and map those form fields straight through to the corresponding table columns. For the keyboard–switch association, it was of course necessary to provide a one-to-many association, but this process isn't part of the data entry wizard.

Before choosing a schema, it's important to contemplate the kind of data we're dealing with here. For example, one OEM keyboard can be sold under multiple brands, but the same keyboard may be sourced from multiple OEMs. I've put FCC ID in at two separate levels (family, to normalise it) but also keyboard level, for either where there's nothing meaningful to add at the family level (you can see where keyboards can be linked to make or family) or when the keyboard's FCC ID differs from that of its family (one of Ortek's unending FCC ID typos, or where the OEM customer has assigned it a different FCC ID, which is quite common). Every switch must have a series, which makes the schema cleaner, but all too often that series isn't known (may just be one switch).

Pretty much every possible detail you can imagine is going to be NULL somewhere. Keyboards with no serial, no dates on *anything*, no make, no model, no branding, switches we can't identify, keyboards where family/series is meaningless (such as a built-in keyboard specific to a microcomputer) … What would you do for those fake IBM Model M keyboards with blue Alps switches? Make: unknown. Series: none. Model: I guess we use the model it pretends to be. We do have IC dates on those. In Keycombo, I've programmed in algorithms to try to present something vaguely meaningful as a display name.

Also, the meaning of terms like "model" and "part number" aren't agreed upon. Cherry's naming and numbering system is at odds with other companies. Let's say you've found a G80-1866HAD with an OEM label that has no model number — what model is this? I've made it a strict rule not to enter anything that's not written on the keyboard. This can be classed under G80-1800 series, so it *may* be Modell MX1800 (my G84-4135 is Modell ML4100), but model is NULL as the label doesn't give a model and we can't just make something up if we don't know! However, Cherry article numbers are not part numbers, which are separate (albeit rare). In fact, a lot of old Cherry keyboards have no model numbers at all: pretty much any G80-0nnn has no model number, so what do those have for their display name? I was putting Gnn-nnnn as the part number but that's wrong.

Some of what you see in that schema was a desperate attempt to wedge in this mess into something manageable. I realise that some other fields (e.g. date records) are also possibly better suited to one to many as that would actually greatly ease the data mining (notice how, on the SKCM page I cited, the switches are listed top to bottom in order of earliest recorded date) but again, this was done out of UI limitation, which is why I indicated that the UI is critical, as there's just so much data entry for every keyboard, and every one-to-many property complicates the UI. Again for serial numbers as you can have more than two of those! (I realise that if people want to just store their own collection, then most keyboards will be entered with far less data, but to achieve my own aims, there's a lot to enter, without considering how much else we could store, such as country of origin, keycap type, layout, protocol, plug, colours, condition, purchase price, catalogue price, places to buy etc.)


Besides, I was primarily asking about the name and whether it's legitimate or desirable to host this at Deskthority.

[Daniel Beardsmore]

If this were to become a feature of Deskthority, then the name would want to be a single word. Alongside the Deskthority wiki, you'd have the Deskthority something. If it's to be independent of Deskthority then I'd expect the name to be a bit more catchy than "KBDB".

I see lots of potential for integrating such a keyboard database with the Wiki.

OK, I'll try to keep this fairly short, since I've already killed the topic stone dead. In an ideal world, we'd replace the wiki outright with an object store. Specific classes (e.g. keyboard, switch, company) would get a class property to hold wikitext for their descriptive content. Writing a wiki formatter is not a huge task, but it would be more interesting if you wanted proper revision history (and would this be at the object level, or specific to the wikitext fields?) You'd not need to recreate MediaWiki's gruesome template system, as infoboxes would simply be view definitions on the objects in question.

By the time you've assembled an object store to hold what is a substantial portion of the wiki, it would seem more sensible to simply add a class "Article" and store the wiki content within this new database where it can fully integrate.

The basic premise of this idea is objects have properties and views. Default views include "show as its own page" and "show as an item within a list". For example, a keyboard would appear in a list as a hyperlink by default, whereas class GalleryImage would have a custom view defined for list items, being a thumbnail as well as its own-page view. (I guess each class would also have static views for "begin a list" and "end a list", as that won't always be <ul>…</ul>.) A keyboard would define its "show as its own page" view to include its title, its descriptive text, its gallery (a property of type GalleryImage one-to-many, displayed in list view), and an infobox (itself, displayed as a custom "infobox" view). (Infobox view would, I guess, need some kind of static template — this falls outside of my mental model!)

I see 2.5 levels of implementation.

Level 1 is a wholly custom system designed just for us.

Level 2a allows for the objects and classes to be entirely defined in the admin panel (with the SQL DDL statements invoked to dynamically create and alter the object tables) — this would be a generic application for any hobby based on large amounts of objects (whether that be trucks or butterflies) that could see life well beyond the scope of keyboards. Our database would be an application of this software.

Level 2b makes it a totally generic object store CMS (this is the original idea I had long before I got into keyboards). On top of this, you'd customise it a bit to make it more centralised to hobby encyclopaedias: this would be effectively a pre-packaged site with the image galleries, article categories, articles, and so forth already defined in the default database. This software would even permit creating a forum — I envisage that you'd be able to key in the basic forum/subforum/topic/post structure and have it able to display topics and posts within 15 minutes, as in effect all a forum is, is a hierarchy of simple classes with multiple views (e.g. show a topic as a page, or show it as a list item within its forum) and paginations. (User accounts, groups and sign-ups would be part of the base software.)

(I'd explain this in more detail but I've already exceeded the waffle limit by a factor of 3–5!)



Short short version:

My objective is very simple: record keyboards, and generate graphs of the years in which switches were found. There's no article text, no images, no ownership, just companies, keyboards and switches.

You can extend this in so many ways, and if you don't stop extending it you'll end up replacing the entire wiki and the forum outright (and give 7bit his own marketplace to play with while we're at it).

So where do we stop? And from this we would deduce the bigger question: how do we start?


The other question of course, is who's responsible? I would not be surprised if I get voted out. I don't know what my role in this is going to be — dictator, minion, no idea. It's all in the air at the moment.

I don't think there any easy answers, and we may not reach any conclusions for a while.

[elecplus]

I would LOVE to see this happen, but NOT to replace the wiki.
Of the 2800 or so keyboards I have in stock, the wiki has been super useful. The people here have been more useful, TBH.
Not all Dell AT101W have black Alps.
IBM buckling springs appear in everything from AT&T to Dell to Compuadd (which even has a BAE).
Not all Wyse have vintage black MX; some have green Alps.

You would have to come up with something idiot-proof enough to be able to look up "striped ambers" or "lipstick".
Something that can start with a shape (rectangle) and lead you through a process to help identify what it is.

[Daniel Beardsmore]

By "replace the wiki" I mean make something better than the wiki.

When you say "rectangle", do you mean keycap mount or keyswitch? For switches, see [wiki]Switch recognition[/wiki]. For keyboards, we don't have a taxonomy yet, but that would be desirable. The issue with keyboards is that most of them are basically rectangular, beige and with 101 or 102 keys, but you could classify the other layouts.

[elecplus]

There are one-handed, ortho-linear, and foldable, not to mention strange things like Maltrons.
Switch recognition makes me feel rather dense in the brain. I know most of the switches, and then I come up with something like "clicky Aristotles" which were supposedly never made. Or Focus 2001, which do not contain rectangular switches, much less Alps. I know I have weird keyboards, but I suppose you want all these weirdos documented? Maybe I should send you a rather large parcel? XMIT has looked at most of them, and he is as stumped as I am.

You might also consider a section for discreet rubber domes, since these do not feel like mush, and are actually rather nice to type on. Plus they are quiet for the office.

[Daniel Beardsmore]

Where did you read "clicky Aristotle"? All [wiki]Taiwan white and black shaft[/wiki] switches are clicky including Type 6 "Aristotle" (who likely made all of them). The only non-clicky switch associated with Aristotle is [wiki]Aristotle Yellow[/wiki] which is linear. Focus FK-2001 should only have Alps-shape switches (Alps/Himake/Omron) — I assume you've found something new in one of those?

There aren't many keyboards with discrete rubber domes. Alps integrated dome is one type, but they're not all that common and they haven't been made for many years. The Alps-made AppleDesign Keyboard is another, which is rubber dome over membrane. There is some company that tended to use discrete domes — we have a modern 2000s era black keyboard in the office with discrete domes (I thought it was Silitek, but the Silitek keyboards on the wiki are rubber sheet). Normally though the domes are either glued to the top membrane (as with older NMB keyboards) or moulded from a single sheet of rubber (pretty much everyone else). It's true that my existing database doesn't offer much in the way of switch classification and it's something worth having in the new database. However, this topic seems to have already fizzled out, and we get to toss a coin to see which one of us will be switching off the lights.

You wanted to send me keyboards? I stopped collecting them as they take up too much space, and I can only photograph them in the summer on sunny weekends: I use the front garden path, but the sun must be high enough up to clear the trees otherwise I get shadows right across where I'm trying to work. Nothing else ever works for keyboards. Loose switches, I do collect, though, but I have a huge backlog of unprocessed photos and no great motivation to trudge through them as they all suck. (I generally try to get NOS switches as they look far better in photos.)

You're welcome to start an "Identify this switch" topic and post loads of photos for people to identify.

[Scarpia]

Level 2b makes it a totally generic object store CMS (this is the original idea I had long before I got into keyboards). On top of this, you'd customise it a bit to make it more centralised to hobby encyclopaedias: this would be effectively a pre-packaged site with the image galleries, article categories, articles, and so forth already defined in the default database. This software would even permit creating a forum — I envisage that you'd be able to key in the basic forum/subforum/topic/post structure and have it able to display topics and posts within 15 minutes, as in effect all a forum is, is a hierarchy of simple classes with multiple views (e.g. show a topic as a page, or show it as a list item within its forum) and paginations. (User accounts, groups and sign-ups would be part of the base software.)

Great minds think alike. That's very like the "side project" I am working on Data Model First is my mantra.

My approach may be a little bit different, but not by much. I have designed an information architecture specification language and I am working on a 'compiler' (*) to turn a given spec into a web application. It's easy to do in theory, but I think the real challenge is in defining and sticking to a limited scope, choosing which features to cut and where to go with convention over configuration, since one could easily end up just reinventing the ORM paradigm and adding a form builder.

Do you have any thoughts regarding language/stack choices? I would personally lean towards a MySQL/PHP/Laravel based stack since that has the benefits of:

1. widespread adoption both in active developer base and (affordable yet stable) hosting options
2. free and open source means flexibility and usually a higher quality of code
3. best practice framework in terms of ORM, templating engines, DB migrations and MVC patterns
4. good to great performance characteristics for full-featured web applications (**)
5. huge catalog of community-built modules and extensions

Anyway, if and when my platform is ready for action, I'd love to grind its teeth on something like this; however, as all one-man projects there's an inherent risk of it becoming vaporware, which is why I'm reluctant to commit to anything. But I'd love to help in any way I can. Beardsmore certainly has the right idea in my book

Spoiler:

*) I put 'compiler' in quotes since I am getting the lexer & parser parts 'for free', so I am really starting from the abstract syntax tree

**) I know many people would argue with my performance statement, noting one or more of the following:

• that Laravel is slow (true for naïve hello world benchmarks, a lot less true when using equivalent middleware, and entirely irrelevant when you apply proper caching and consider the far more critical front end performance)
• that PHP is slow (the go-to excuse by junior magpie developers who think a faster/better/cooler language will fix their shitty O(n^2) data retrieval endpoints; the truth is that essentially every language is fast enough unless you are writing shaders or GPU-optimized neural networks or have special needs like low-level memory control or built-in concurrency primitives. And with PHP 7 we even get a lot of the benefits of Facebook's optimized JIT compiler from HHVM. If PHP is fast enough for Facebook, it's fast enough for you.)
• that MySQL is slow (again, MySQL itself is fast enough for real-world problems, unless the dataset for any one query reaches terabytes and/or you need it to play nicely with Hadoop, or if your use case absolutely requires schemaless records or window functions, or if your use case calls for a columnar store or a dedicated graph database. More importantly, if you do reach a scale where MySQL doesn't quite cut it, you can drop in a high scale, high performance replacement like MariaDB, Percona or Amazon Aurora)

[Daniel Beardsmore]

MySQL has its warts, but what doesn't? It can't comment on PHP as I rarely use it and I don't stay up-to-date on language features (nor is my web host going to have a cutting-edge version) — while my own website uses PHP, I normally use Perl, which I find hugely preferable. I've never used any kind of ORM system at all — it's not ORM that I'm interested in specifically but simply something that ensures that the SQL schema is updated appropriately.

Although I went through the whole process in my head of creating a forum, I forget now just how I had it working. As I recall, I've always visualised it this way around (with properties capitalised to save using markup for emphasis, and the syntax is whatever I just made up today, as it's not set in stone):

1. Log into the admin interface
2. New class "Post", properties Title (string), Message (string) (object creation and modification metadata is automatic and does not need specifying)
3. Grant group "Users" creation rights on Post
4. New class "Topic", properties Title (string), Posts (Post, collection of) (some heuristics may be in order to derive the actual direction of the foreign key if a class A (e.g. Post) is used solely by class B (e.g. Topic); here, the Post class will get a Topic (pseudo-)property)
5. New class "Subforum", properties Title (string), Topics (Topic, collection of) (not going to bother with Deskthority's subsubfora for now!)
6. Define view Post:In topic:Item as a table row showing its title, message, author and date properties
7. Define views Post:In topic:List start as table open and Post:In topic:List end as table close respectively
8. Define view Topic:Page as the <h1>{=$self.Title}</h1>, then an iteration using view "In topic" {#$self.Posts:In topic,paginate=20,sort=Created on DESC} (which will call the Post:In topic view on each item and wrap with the List start and List end sub-views), and finally request class Post's native view "Create button" which will open the native object creator <p style="text-align: center">{=Post:Create button,set Topic=$self}</p>
9. Define view Subforum:Page as an iteration of topics, as above, but using native link-to view for the topics (points to a URL that gives the Page view) since a basic bulleted list is enough to see working results: {#$self.Topics:Link to,sort=Created on DESC,paginate=20} — although for that to be true, it stands to reason that "Title" would have to be a native property otherwise the native iterator wouldn't know how to caption the link
10. Define view Main:Page content as an iteration of subforums {#Subforum,sort=Title ASC} (here, we just list the entire class; also, Main:Page will have the meta aspects such as register an account, login etc — we only want to override the content area for the purposes of this exercise)
11. Enable public account registration
12. Manually create the initial subfora in the admin interface, since they seldom change

That's almost it. You can now register an account (native functionality included in Main:Page view), open a subforum, and click the "Create Topic" button, and then "Create Post". There'll be a hook required to get the topic creation to create a post at the same time — this would start with an override of the Topic:Create view.

It's not free of computer syntax, but it's free of any Turing-complete language to achieve a very rudimentary forum. The use of a Turing-complete language will be required of course, as any realistic site above blog-level will need it for something or other — starting possibly with hooks, such as On validate, Before create, After create, Display as ($view) etc.

Other things to define include declarative field validation using built-in types and regular expressions, and maybe a baby-regex like in Acorn Screen Editor for more common cases — this will be obeyed both client-side and server-side.

AcornScreenEditor.png (13.29 KiB) Viewed 32281 times

(For an 8-bit machine it was a pretty awesome text editor; it also included a BASIC tokeniser/detokeniser, as seen under Shift+F4.)

For a wiki image, you'd set the Page view to be a zoomable image + metadata, and then Gallery:Item, Gallery:List start and Gallery:List end views to give you {#$someImageGroup.Images:Gallery} — the :Item subview would have the image scaled to a set size plus the caption and a link to the full-size image, again all with declarative markup only.

These class definition collections are what you'd package up as pre-built sites, and you'd be allowed to import and export them, so that you could add multiple sub sites to one single site.

I'm not sure that this is anything even remotely like what you had in mind.

For what it's worth, the idea comes from Allaire Spectra. It's Allaire Spectra without being saddled with ColdFusion or compelled to name everything with GUIDs — cross-project implementations (e.g. several sub-sites that want to interact) would need to use namespaces to refer to each other, but I still refuse to use GUIDs as they're utterly abominable for anything at a scale lower than an entire OS or if you're employing a Soong-type artificial lifeform.

[Scarpia]

Ah, that makes sense. Yes, I believe the aim is mostly the same, but I guess our approaches are quite different. I never used Spectra (or ColdFusion), but I did use Drupal with CCK for a little bit back in 2007, which is where I took some of my inspiration (the UI-based data model creation you describe is very similar to how CCK worked in practice, although in Drupal the views would have to be created separately IIRC).

As a sidenote, I actually went for the same idea at first (a UI builder) and managed to hack together a proof of concept which created a meta-controller and meta-views which would auto-detect and apply any database table, but it wasn't expressive enough for my liking. Basically, the more I worked on it, the more it turned into an expert system for developers only, which I felt defeated the purpose of having it built into the CMS itself.

Spoiler:

Specifically, I realized:

• I needed to handhold field-based authorization (since you don't want your system to autogenerate an endpoint for a DirectMessage type that allows the sender to change the fromUserId field)
• I didn't want to allow the CMS to make code changes since that would break things if you ever wanted to launch multiple web servers behind a load balancer (unless you built the CMS itself as a git client, which I wasn't willing to do). This meant having things like controllers and routes and models defined inside the database, which is doable but can get messy, and is a real challenge to optimize since you don't want to make DB calls just to do your routing.
• I couldn't do the kinds of data transforms I wanted in a way that users would intuitively understand (e.g. one endpoint returning a full blog post, and another endpoint returning just the blog post title, slug and thumbnail).
• That it became convoluted to customize the admin CRUD look and feel.

That's why I went back to the drawing board and wound up with a specification language and actual code generation. The major drawback of my approach is that it's really a one-time setup, and the moment you manually alter the generated code, you really can't run the compiler again without breaking things. So my system is conceived as a bootstrapping tool only, a time saver at the start of a new project.

Your system, in contrast, would allow arbitrary new data models and features to be created on the fly on a production site, which would be frankly awesome.

[Daniel Beardsmore]

So, as a summary:

1. [#]Without knowing more about what you implemented (having never tried to implement anything like this myself) I don't know that my idea wouldn't end up in the same boat as what you already tried to create, as nothing works in reality the way it ought to — for example, does my declarative markup-based arrangement resolve your endpoint issue in spoiler bullet 3? (Some means will be required to attach code to views too, possibly one text box for code and one for markup, with the latter accepting variables set by the former)
   [#]I don't know where we stand in terms of what if anything would be used to implement the keyboard database.
   [#]The very brief spark of interest in the keyboard database has faded away, leaving it back to square one where we don't even have answers to the basic questions posed originally.

The initial interest level really surprised me, but it didn't last.

[Scarpia]

I think people are still interested, just not everyone likes to blurt out ideas as much as I do

[seebart]

The initial interest level really surprised me, but it didn't last.

You posted this a mere three days ago, patience.

[Daniel Beardsmore]

You're new here, right?

[snuci]

No loss of interest. You guys went from ideas to the architecture in implementation. I can't add to that discussion.

In fact, I was trying to think of a way to use this for vintage computers There are much more parent/child relationships with that, however.

[Daniel Beardsmore]

Yet virtually no input on the actual questions. Realistically this would never be anything more than entirely ad hoc code or built on some existing framework — there won't be any shiny new object store system ever, so it really does come down to whatever off-the-shelf system sucks the least. No-one really wants this database that badly, not even enough for me to feel like it's worth trying to salvage the one that already exists.

[mecano]

Hello,
I guess it all boils down to what you are trying to achieve, sharing phpBB credentials is on a technical level trivial.
KBDB is a nice name indeed.
Having a separate entity with it own domain will maybe make people not interrested (at first) in joining a community forum to take the plunge while it'll surely benefit from sharing credentials with Deskthority and still being under it hood.
If it has to be, subdirectory for me, it makes more sense.
Knowing drupal CCK/Views, yes it can apply here perfectly, you need content revision as well and solr as Scarpia said.
Basically create content types:
- keyboard,
- switch,
- etc.
For each content type create fields:
- keyboard : name, image, brand, family, switches, year, etc.
- switch : name, image, brand, family, type, year, (used in) keyboard, etc.
note that you can have keyboard variations under the same name then group all variations on an index page or a content block.
Now link contents by fields, ie if keyboard 'A' switch field name is equal to switch name contents will be linked to each other, same for brand, etc. You can do that in Drupal automagically.
Give the ability to users to perform search through filters and facets.
Moderate revisions when users are editing or adding new content.

[Daniel Beardsmore]

If it was to live at Deskthority (assuming that is even permitted) I'd like a two word name, "Deskthority something", like "Deskthority compendium", but using a word that's easier for non-native speakers to deal with. "KBDB" is a bit of a mouthful. The sad thing is, I can't remember how to pronounce that in French any more. ("wiki" is nice and short, but while French has a "w" sound, a lot of languages don't!)

I don't think you understand the data model at all. A keyboard may have switches from different families, e.g. Alps integrated dome for all keys except lock keys, that are SKCL Green with LED fitted. The switches may differ between instances of the same keyboard: Chicony KB-5181/2 and NTC KB-6251/2 came with a whole variety of switch types. Keyboard–switch is a one-to-many association where each mapping carries the purpose of the key in addition to the type of switch (space/enter, lock, function keys, etc). I even sub-divide switches by sub-type, e.g. long/short black/grey/white switchplate, logo/no logo etc so that we can have clear evidence about the age ranges of these features. I don't even want to think about categorising Series 725, but fortunately I can't as there's no consensus on their subtypes yet.

Ideally you'd want switch selection to be done using an intelligent filter panel, so that you can find switches readily, as the list soon gets very long. You'll also want to be able to create any and every part of the (keyboard OEM + keyboard brand)→keyboard family/series→keyboard→switches←switch series←switch brand chain during data entry, rather than having to keep starting over and over again every time you realise that the next thing you want to enter isn't there. And there's the question of image uploads.

Your description suggests a very weak data model. "ie if keyboard 'A' switch field name is equal to switch name" — this sounds appalling. This should be a foreign key in the SQL database.

Also, as I said, everything and its dog are going to be NULL. Keyboards with no make, no series, no model, nothing but some meaningless code on the PCB. So drill-down by keyboard or series won't work. If you require a series, then you'll have tons and tons of "no series" series, where we have nothing to enter. My model above permits a keyboard to have an OEM regardless of whether a series exists, but that's a horrible mess in itself.

Most of this already works in my live Keycombo database, but the UI is so bad that I've never let anyone try to use it, and it's written in PHP+mysqli, which is horrible. PHP I can live with, but mysqli is a complete abomination. I don't even have transactions in Keycombo, so any crash/glitch leaves the data in a mess: I can't get mysqli to handle errors properly so I have no way to meaningfully detect them and roll back a transaction. (Absolutely no such trouble in Perl's DBI, which works perfectly, but my site is PHP and has been since 2003.)

Keycombo fits the "build one to throw away" pattern: it's a proof of concept that faded out before it ever accumulated enough data to be truly useful.

[codemonkeymike]

PHP Data Objects (PDO) have existed for about 12 years now, they could be used. I think mysqli has been deprecated for a while now, I didn't even use it in college about 5 years ago. http://php.net/manual/en/book.pdo.php

[Daniel Beardsmore]

I think of PDO as "paedo", but yes, I've looked at PDO and it's exactly what I'd expect it to look like. I've never seen any mention of mysqli being deprecated or even much in the way of the fires of Hell flaming it deserves.

[codemonkeymike]

I stand corrected mysqli is not deprecated, that being said PDO would most likely get all the love by the devs as it supports 12 DB's rather then one. I come from a Postgres background only so I have never had the ill fortune to use mysqli