Category: Surfulater – Tech Info

Surfulater Pre-Release version 2.91.0.0 is now available. This has been the longest gap between releases that I can think of, and the greatest overall changes to the code base for some time. This release includes the ability to add Tags to Articles and view Articles based on their Tags. It also includes the start of a move to a new embedded SQL Database engine. The last few Blog posts have largely been about tagging and are worth a read if you haven’t already done so.

We have done this as a pre-release to elicit your comments on the new Tagging system and to resolve any issues prior to a full release. This release sees changes in a broad number of areas and although we feel we have tested most every aspect, real world use will help sort out anything we may have missed. We also want to resolve a few more outstanding bugs and possibly add some more new features before a full release. And most importantly we want to get your feedback on the new Tags implementation.

This release is backwards compatible with previous releases. You should backup your Knowledge Base files before using this release, in addition to performing your daily backups. A third file which contains the new SQL Database is now used with each Knowledge Base. As we move forward, the current three KB files will be merged into a single SQL Database file. This will also enable backups to be done with Surfulater running as well as making for an even more robust database system.

A new Tags field has been added to each Article template, except the Notes template. Simply add your tags here and they will be added to the database. Any duplicate tags or invalid characters are removed when the article is saved. Auto-suggest makes it quick and easy to enter existing tags. 

The new Tags Tree displays all of your tags along with the articles associated with each tag.

  You can also manage your tags here using the Tags Tree context menu.

Tags can be renamed and merged with existing tags, deleted and new tags added.

You will notice that the sample Tags Tree above includes some Tags nested below the top level tags. These are tags which are in the articles in the top level tag. For the tag Borneo, one or more articles with this tag also include tags Asia, Sabah and Travel. In other words the nested tags form an and relationship. Their articles contain Borneo and Asia, Borneo and Sabah and Borneo and Travel respectively. This lets you easily see and navigate to interesting relationships between tags. The best way to understand this is to use it.

In addition to the inclusion of these related article tags, I have been thinking of enabling you to specify  Tag search expressions, which would appear as virtual Tags in the tree. For example you could specify “car and ford and not 2006″ and all matching articles would be displayed. This may well be added in the future, but first I want see what folks think of the current implementation.

I know many Surfulater users have been eagerly awaiting the ability to Tag content and I very much look forward to hearing what you think of this first release with tagging.

Full release notes are in the Surfulater Help and a new topic: The Basics | Tags and Tagging Articles has been added. Known problems with tagging are in the release notes. Note that some keys have changed in this release. See the release notes for details.

Click here to download this pre-release and when you have used it for a while do let us know how well it meets your needs, or doesn’t!

This is a quick update on how the new Tagging system is coming along, and to show you the new Tags Tree. Without further ado here it is:

The first thing to note is the new Tags Tree button on the Knowledge Tree bar. The Tags Tree presents each Tag as a Folder, which in turn contains the Articles which include that Tag. Like the Knowledge Tree you click on a Folder and see all the articles in that folder in the content window, or select and display specific articles.

Next you can see the same article tagged into multiple folders. For example the article titled CodeProject: CppSQLite … is tagged with C++, Database, Library and SQL as shown in the next screen shot. If you look back at the first screen shot you will see three of the Tag folders expanded and this article in them. The fourth tag Library, is collapsed.

 

Editing an articles Tags in the content window, updates the Tags Tree accordingly. For example if you delete the tag SQL for this article, the article will be removed from the SQL folder. Similarly adding new tags to the article, will make it appear in the folders for those tags.

You can also edit the Tags themselves, either in the Tree or via. their Folder in the content window. When you rename a tag one of two things happen. If the new name doesn’t already exist, the tag simply changes to the new name and the tree is updated. If the new name does exist then all of the articles with the original tag are moved to the folder with the new name, and the original tag is deleted. Where an article has both the new and the original tag, the original tag is deleted from the articles tags field. A prompt is displayed allowing you to Cancel a rename where the new name already exists, preventing the articles from being moved and merged.

The ability to rename and merge tagged articles is an important one. For example if you had created the tags cattle and cow, used these these in lots of articles and then realized this was a mistake and you should have only used cattle, then all you need to do is rename the tag cow to cattle.

Tags can also be deleted via. the new Tags context menu.

The design of the the Tags system is such that rename and deletion of tags happens very quickly. In the case of  renaming a tag to a non-existing name, the entire knowledge base is updated instantly, even if that tag is used in a large number of articles.

As mentioned in recent posts the new tags system has been implemented using the a new SQL Database Engine and so far, I am very pleased with how this has come together and is how well it is working. But it is early days and so far only Tags use the new database. The design work for moving all existing Knowledge Base content to the SQL Database Engine is progressing smoothly.

There is still a bit more code to write and more testing to be done, but a release with Tagging isn’t far away now.

It is time to do a bit of show and tell about the new Tagging capability being added to Surfulater. The ability to add tags or keywords to articles and then locate articles for specific tags, has been a reasonably common user request. Tagging of content has become extremely popular in recent times, especially on Web sites. I have done a fair amount of research into tagging, folksonomy, ontology, semantic web etc. over the post few years and it is clear there is a lot happening in these areas and we are in many ways still in the early days.

A problem with tags is that single words on their own, are often ambiguous and have no context. For example what does “disk” mean? Is it a hard disk, a record, a CD, a disk in your back or maybe even a plate.

CamelCase (or WikiWords) offer one solution to this, but one I don’t personally like, not that I’ve actually used it much. To quote Wikipedia

CamelCase is the practice of writing compound words or phrases in which the words are joined without spaces and are capitalized within the compound. ex. HardDisk … CamelCase is rarely used in formal written English, and most style guides recommend against its use.

Hierarchical tags are another way to add extra meaning. ex. hard.disk For some time I thought that the ability to use hierarchical tags would be worthwhile, but have in the end decided against that.

To locate articles using tags, my current plan is to enable expressions to be used to specify tag groups. For example: car AND porsche. A tag hierarchy would use car.porsche for this. The problem here is if someone tags porsche instead of car.porsche, then the former won’t appear under the car.porsche listing. It is also more effort to enter hierarchical tags. Expressions offer endless flexibility here. You can find just “car”, just “porsche” “car and porsche”, “car and porsche and not 911” for example.

A tree will be used to show all Tags (as folders) and their matching Articles as children. The aforementioned expressions can be used to specify virtual folders in the tree, along with the matching articles. This will be a new tree in the Knowledge Tree bar.

Until this is all in place and in use, it is difficult to say how well it will work in practice, but I feel it is a good solid starting point.

The next issue is that the use of Tags is a bit of a mess, everyone does their own thing, and there is little standardization, which of course makes using them to find things is less than ideal.

To quote from Wikipedia again:

There are drawbacks to such tag-based systems as well: no standard set of keywords (also known as controlled vocabulary), no standard for the structure of such tags (e.g. singular vs. plural, capitalization, etc.), mistagging due to spelling errors, tags that can have more than one meaning, unclear tags due to synonym/antonym confusion, highly unorthodox and “personalized” tag schemas from some users, and no mechanism for users to indicate hierarchical relationships between tags (e.g. a site might be labeled as both cheese and cheddar, with no mechanism that might indicate that cheddar is a refinement or sub-class of cheese). Services which allow both tags and folders for organizing bookmarks (such as Netvouz) make this less of a problem though. 

I’ll add Surfulater to that list. It goes further by enabling an article to be in many folders at once and letting you add links between articles, to build a web of related information.

In this situation all that folks like me can do is provide the best tools we can and hope they are used wisely. To this end it is important to use a well defined set of tags (see controlled vocabulary), instead of any old ad-hoc words. To help ensure consistent use of specific words I’ve implemented an auto-suggest list which displays words matching the characters typed so far, which you can then pick from. The first image shows the auto-suggest list with matches for the letter d, along with the matching characters bold and underlined as a visual cue. The second image shows the selected word entered as a tag.

  

You can also press the Down Arrow key to access a list of all defined tags, as shown here.

 

When you enter a tag word that isn’t present in the tags list, it is automatically added. Next a window opens showing the newly added tags, enabling you to correct or delete these tags as required, as well as add a description. If you are happy with the newly added tags just click “Close” to continue. The design here is to try and impinge as little as possible on your work-flow. An alternative approach would be to prompt you each time you typed a word that didn’t exist and ask if you wanted to add it. This could be quite tiresome, especially during the early stages of building your list of tag words.

As I mentioned on the blog recently the inclusion of tagging has dictated extensive changes to the underlying database engine that Surfulater uses, as  well as several other fairly major framework enhancements. This means that the time to the next release will be a bit longer than usual, but hopefully well worth the wait.

We all want access to our information wherever we happen to be, assuming of course there is a PC available, or maybe even a Smart-phone. Various technologies are making this possible, all the more so as our lives become “always connected” to the Net and as data speeds continue to improve. There is interest in moving content out of Surfulater for access by other programs and for viewing it when Surfulater isn’t available. For the former Perry Mowbray has written SUL2Text which extracts the text from a Knowledge Base and dumps it into a text file.  And Perry also wrote HTMLView which generates a Surfulater TreeView and Articles. You may also be aware that Surfulater’s main content is stored in an open standard XML file, which can be accessed by other applications.

If you read Perry’s HTML view article you will see I sparked his interest, as I’d thought for some time that the ability to view Surfulater Knowledge Bases in a Web browser would be a very, very useful. This capability delivers important new ways to access all of the valuable content you’ve put so much effort into collecting and most importantly does so without any need to have Surfulater installed on the PC you want to view it on.

Without further ado here is a screen shot showing content from the MyKnowledge example Knowledge Base in Internet Explorer.

As you can see we’ve got the Knowledge Tree and the Content window, just like in Surfulater. Clicking on any folder or article in the tree displays its contents, and the keyboard cursor keys also navigate around the tree. The tree is expanded to match how it was in Surfulater when it was published.

To play with a real live demo click here. Note that Javascript must be enabled in your Web Browser and this being an alpha-release there are some known problems. Depending on your Internet connection speed and the phase of the moon, this will take around 10 seconds to load. We will likely be able to speed it up a bit in the final release.

Noteworthy features in the demo include the ability to hide or show the tree by clicking on the [< <] button.  When hidden, you can click on the vertical bar to auto-show it or the [>>] button to lock it open. You can resize the knowledge tree panel by dragging its border and you can right click on a folder to access its context menu, which in turn lets you expand and collapse all of its folders. Capabilities like toolbars, more menu functions etc. will likely be added in the future.

By now you may be thinking this is all well and good but what would I use it for. I see three main use cases. First you can publish your content to a PC on your LAN so anyone on the Network can view it. Second you can publish it to a USB Memory Stick or USB Hard Drive and then access it on any PC, wherever you happen to be. And third you can publish it to a Web Server enabling anyone with an Internet connection to view your knowledge bases.

This new publishing capability sets your information free, enabling you to view it on any PC, without any need for Surfulater to be available on the PC. It takes our open XML format Knowledge Base a step further, allowing you to share your information on the Web or a LAN, or carry it on a USB Stick, so you can see always see your content, wherever you happen to be.

Once your Surfulater information is accessible from a Web Browser, it’s not hard to see the potential this offers for the future.

One of the most requested Surfulater capabilities is the ability to select multiple articles in the Knowledge Tree. In hindsight it is clear that omitting this in the original design was a mistake. When you develop any type of product you have to work through an array of trade-offs, otherwise the product may never reach the market and will likely be cost to much. Opting not to include multiple selection was such a trade-off.

Leaving this out was a mistake for two reasons. First I didn’t realize how much our users would want to work with multiple articles and second it has turned out to be quite a chore to rework the code to implement multiple selection. It sounds simple enough on the surface, but it is quite complex once you dig down a bit.

Selecting multiple items is reasonably straightforward, apart from Ctrl+click which toggles the selected item. However the user interaction that follows is complex and tends to vary for each type of action. For example if you select three articles, right click on one and choose Copy then right click on a Folder and choose Paste what should happen with the selection. In the version of Surfulater you have now the selection moves to the pasted article. In the next version it keeps the selection on the original three articles. This brings it in line with how other Windows programs, such as Explorer, behave. But what happens when the original three articles are no longer selected when Paste is used – does the newly pasted article get selected or does the selection stay wherever it was?  And Paste is just one of a range of actions and associated outcomes that can be performed on a multiply selected set of articles.

Working though the possible permutations for each command is tedious and deciding on what is the most desirable outcome isn’t necessarily straightforward. And what I think is desirable you may not!

I’ve also changed the behaviour of Right mouse click for the next release so that it no longer changes the current selection. These and other changes make it easier and more intuitive when working with selected items in the Knowledge Tree.

Bit by bit I’m working on making it easier for folks to create their own Surfulater article templates. Once I’ve finished laying the groundwork and updating the existing styles and templates I’ll be ready to put a release out and write a strep-by-step guide to making your own templates. This is all a precursor to having template editing capabilities built-in to Surfulater.

The first step in this process has been to review the styles that are currently used by the templates and ensure they are sufficient to cater for the varied templates you are likely to want to create. This requires some guess work on my part as my crystal ball is currently out for repairs. With some guidance, I’ve refined and I’d like to think improved on the current look. By styles I’m referring to things like borders, fonts, margins and the like.

These images show what I’ve produced so far. The main differences you will see include a thinner border around the record, thinner more subtle borders between cells, the use of different colored borders, a different and smaller font used for field labels, and the focus rectangle and edit background color going all the way to the edge of the cell.

New look – plain

Current look – plain

New look – with focus

Current look – with focus

New look – editing

Current look – editing

The differences are subtle but I feel offer a cleaner and tighter look. I’m undecided about changing the color of the field label text. Of course this is something you can change, but seeing that most people won’t bother, a sensible default is needed.

These screen shots show a Folder, which is all I’ve worked on so far. However I’ve updated the code so that article and folder templates can use different styles.

The final screen shot I want to show is with the Content Elements set to hide Field Labels and Edit Pencils. With these settings in the current release you loose ability to collapse and expand articles as the [+] [-] image disappears. It also isn’t all that clear where one record ends and the next one starts, when you are viewing all articles in a folder. To resolve both of these issues I’ve added a new narrow column that includes the expand/collapse image and runs the full height of the article. This color alternates from article to article, making the visual separation much clearer.

I have to say that I find writing CSS and HTML a frustrating experience. It just isn’t something I do enough of, for it to become second nature. I’m also graphically challenged, which doesn’t help. With that in mind if anyone knows of any designers with a keen eye and good CSS skills, I’d welcome being put in contact with them.

From here I’ll be enhancing the template system to enable you to use multiple templates for each individual type or category of template. For example you will be able to have say three different templates with completely different layouts for the Web Template and switch between them at any time. Existing articles will be displayed using whatever template is selected at the time. When a new article is created that uses the Web Template, the current template from the three available will be used. This not only lets you have a custom layout and look, but more importantly it lets you customize the fields that are used. For example you may want to add some new fields and remove or rename some of the default fields. All with the flexibility of switching templates for content that already exists.

Ok I’m done. Do let me know what you think of the new styles I’ve shown above. Maybe you like things just the way they are now!