In my many years of developing software I can’t recall a more contentious issue than whether data is stored in the cloud or not. This is evidenced in the comments (1, 2) to my recent blog posts and in other communications.
People fall into two camps, they are either strongly against having their data stored in the cloud or they are quite happy to have it there and look forward to the benefits therein.
The former group sites concerns over security of their data, a feeling of loss of ownership, the inability to access their data if they don’t have an internet connection and concerns about the cloud provider going out of business, which are all legitimate issues. As for security, strong data encryption makes it very difficult for others to access. The company going out of business is an issue regardless of whether data is in the cloud or not.
On the flip-side the benefits for applications and content in the cloud are clear:
- you can access your data from any PC anywhere in the world.
- you can collaborate and share information with others (if you want to).
- someone else ensures you data is safely and regularly backed up.
- you don’t need to install any software.
- you can use any operating system (Windows, Mac, Linux) on any hardware and use Tablet’s to access content.
- you don’t need to do regular software upgrades. you are always using the latest version of the software.
- you don’t need to worry about your PC failing and being unable to access your information until it is repaired/replaced and software and data restored. Or worse being lost or stolen.
- you don’t need to leave a PC turned on 24/7 wasting electricity and costing you money.
This still leaves one big issue: “the inability to access your data when you don’t have an internet connection”. My feeling is that this is much less of an issue than it was say 5+ years ago with ubiquitous internet access now common place. That said there are still times when we may not be connected, such as in an aeroplane or out and about in no-mans-land.
It remains clear that some people will not use the cloud for certain applications or content. I think this will change with future generations who are used to having everything in the cloud and being connected 24/7.
Taking these concerns on board I see several solutions which I did touch on, on some of my blog comment replies. The main one being the ability to install the software locally and access it as a personal cloud. This gives you full ownership of your content, but loses most of the significant benefits I listed above. By personal cloud I simply mean software that is installed on a specific PC, whose content is accessed from that and other PC’s using a Web Browser or other application.
Another possibility which I personally find intriguing, is for us to bundle up a small low cost computer and the software into one package. If you follow the low cost, small form factor (SFF) computer area at all, you will most likely have heard of the Raspberry PI, which is a basic $25-$35 computer. More powerful SFF computers are available from $50-$100, with these being better suited to my use case. You would simply connect this to you Local Area Network (LAN) either via. an Ethernet connection or WiFi, plug in a USB Hard Drive and turn on the power, and away you go, with access via. your favourite Web Browser.
The benefits of this packaged hardware+software solution could include:
- you can access your data from any PC anywhere in the world.
- you can collaborate and share information with others (if you want to).
- you don’t need to install any software.
- software upgrades could be handled by a service we offered.
- you can use any operating system (Windows, Mac, Linux) on any hardware and use Tablet’s to access content.
- it uses far less power than a conventional PC making it cost effective to leave on 24/7.
- all data is kept on a local hard drive, so you are in complete control.
Again I find this an intriguing solution, one I’d personally welcome, however I have no idea what level of interest there would be in the general marketplace.
A final possibility is to create a local application that is simply installed as a Web Browser Extension. There are lots of Browser Extensions around that do lots of interesting things, however there are fundamental issues with this approach. The main problem is that the database capabilities built-in to web browsers impose serious limits on the amount of data they can store. Further the current state of browser database engines is a mess with different browsers implementing different and incompatible database systems. And they don’t stack up all that well against fully fledged database systems.
Browser Extensions suffer from other problems as well. We have to write, maintain and support quite different extensions for each Browser. Extensions have a bad habit of breaking, as new browser versions are released. And not all Browsers support Extensions (IE, Opera, Tablets?).
In my ideal world I’d have an application like Surfulater running locally as well as in the cloud. The local version would be used whenever I was unable to access the Internet. It would be automagically synchronized to the cloud application the next time I was connected to the Internet. Conversely all updates done on the cloud application would be synchronized back to my local application. And the local application could be installed at multiple premises in different locations, with them all being kept in sync. This gives me the best of all possible worlds. It is however the most complex and costly solution to produce and deliver. (I wrote about this back in Dec, 2010).
Of course there is even more to this I could cover, but that’s enough for now. As always I welcome and look forward to your comments.
-Neville
PS. In reviewing various related communications I noticed a request for a Portable version of Surfulater (ie. one that runs on a USB stick.) Isn’t using Surfulater as a Web application in the Cloud the ultimate portable version.
Hi, Neville. I read your most recent blog posting with great interest. Your “ideal world” solution is far and away the one I’d prefer. Having my data under my control and always accessible as well as on the cloud seems as close to perfect as I can imagine. I confess that I haven’t been thrilled with the upload speeds (and at times even the download speed) involved in transferring data, especially large amounts of data, to or from the cloud. But that’s my main misgiving about your “ideal world” solution, and it’s not a huge misgiving, since I’ll also have the data available on my computer and won’t HAVE to access the cloud whenever I want to add or edit or find information. It’s the solution I hope you’ll choose. All the other scenarios you propose are quite unattractive to me:
**I agree with you that web browser extensions, with their limitations, their individual differences and quirkiness, and their constant changing, are a royal pain for a developer (and thus for the customers as well).
** As for bundling the software with a low-cost computer–thanks but NO THANKS! The last thing I need is yet another gadget to find room for and have to maintain.
**You’ve already outlined the problems with a cloud-only solution, so I won’t repeat them here. However, I do think you underestimate the danger of the cloud company going out of business. This is not something far-fetched; there have been any number of examples. So instead of worrying only about whether Surfulater will stay in business, your customers will have to worry (with perhaps much more reason) about what to do if the company that runs the cloud on which they’ve stored all their data goes belly up, or, for that matter, sells out to someone unreliable.
As a passionate Surfulater fan, I’m hoping some of this feedback proves useful.
Hi Joan,
I think the term “the cloud” is largely misunderstood by many people, and even my use of the term may not be accurate. For Surfulater Next Gen, initial application and database hosting would most likely be on a Web Server that we run, just like we do now with our various Web sites. We have been with our current Web Hosting company for around 10 years now and I can’t see them going anywhere any time soon. And if there was an issue, then transferring everything to a new hosting company is reasonably straightforward.
Moving everything to Cloud based servers becomes relevant when the customer base grows, as they can scale to meet demand. That said there is even a move to use cloud servers for conventional web hosting and I see that growing.
Cloud based computing is on a rapid growth path and I don’t share the concerns you raise. It is a matter of choosing a reliable provider with a good track record.
Another important consideration is the ability to have your data exported from whatever application you are using, regardless of where the application is running. Many applications, especially Web apps, don’t provide this. Doing something useful with the exported data is another issue entirely, but at least you’ve got it.
As for data transfer speeds, Surfulater Next Gen only transfers content that changes, not entire database files, so performance should be very good.
Thanks for your feedback,
Neville
Hi Neville, I haven’t had a chance to read through your other blog posts again, so apologies if I’m raising something that has already been addressed.
One question for me seems to be how the essential nature of Surfulater is going to change in this new era. For me Surfulater was and still is primarily a way to collect, store, organise and recall website content, using Firefox (and IE, if I had to). I haven’t found any other tool that could do it as quickly and easily. I do have other software for collecting etc. other types of notes that also have web clipping capability but it would never occur to me to use them for that because it is just easier, faster and cleaner to do it with Surfulater. Similarly, I wouldn’t have used Surfulater for organising other types of notes, because I found other software with more suitable functionality.
I wonder if by moving to the cloud Surfulater wants to move closer to being a general note-taker, and what effect that move might have on its web clippings functionality. An unwelcome scenario for me would be to lose the web clippings focus in favour of becoming just another Evernote-type service (which I don’t use anyway).
I can see the appeal of cross-platform functionality. I do have Instapaper and Notebooks for iPad/iPod (and even Evernote) on my iOS devices but I don’t bother using them for clipping website content because they don’t integrate easily with Surfulater, which is my main database for websites. Having an iPod/iPad app might convince me to sign up to a Surfulater cloud, if the essence of Surfulater doesn’t change, i.e. it is still super easy to capture website content, including entire pages as attachments. I’m not opposed to having additional note-taking capability added to desktop Surfulater, but not at the expense of losing the effectiveness of the web clipping functionality. However, it’s unlikely that I would switch all my note-taking and storage to Surfulater, as I’m happy with my other specialist tools such as desktop wikis (and especially not if it results in cloud storage costs).
Regarding the cloud scenarios you mention, I would add a few more. 1) Some people already use online backup services to back up their data daily, which lessens the appeal of a Surfulater cloud regarding the data backup argument. 2) Some apps for note-taking use Dropbox very successfully, which again competes with the need for a dedicated Surfulater cloud service. If I’m already paying for 1) and 2), then I’m less inclined to sign up for dedicated software cloud services.
I just thought I’d share my views with you as as a daily (and happy) Surfulater user.
Hi Dr Andus, I don’t see that turning Surfulater into a Web Application will diminish it’s Web page capture capabilities, in fact the opposite is likely to be true, in that we’ll be able to improve on what’s already there.
As for Note Taking, this is already better than the current Surfulater because it is quicker and easier to add and edit notes. Moving away from folders (see earlier blog posts) to tags also helps here.
Web based backup services like Dropbox are far from optimal for an application like Surfulater. As soon as your database gets to a reasonable size, transferring the entire database to and from some remote server over the Internet will become a real bottleneck, eat lots of bandwidth and seriously impact work-flow.
By contrast the new Surfulater only transfers content that changes, to the remote server. So if you have 10’s or 100’s of thousands of articles and add three new ones and edit four existing ones, then only seven articles are sent to the server, versus potentially 100’s of megabytes or more using Dropbox etc.
This process of adding & updating individual articles happens almost instantly and very efficiently.
Another huge differentiator is that any other users who are using the same Knowledge Base see all changes as they happen in real time, automagically. All of your content is available on any PC anywhere, without have to synchronize using a service like Dropbox. Just open Surfulater Next Gen on any PC and start working.
At the end of the day there is only one way to accomplish this and that isn’t using Dropbox.
-Neville
Hi Neville, thank you for your reply and for the clarification.
Another issue I would like to raise is something another Outliner Software forum member raised recently regarding the evolution of Evernote, which I wouldn’t like to see happen to Surfulater. It could be described as the “dumbing down” of the features that so-called “power users” use, in favour of making it a more widely used consumer app. What seems to underlie such a move is a privileging of cross-platform functionality with a focus on mobile devices, which results in the dilution of desktop functionality.
Even though I do use iPod Touch and iPad for reading and adding content and I could see myself using a Surfulater app to capture websites (and possibly even add my own notes, should there be an advantage for using that over my current note-taking system), my main working tool would always remain the desktop version, and the main point would be to get content into the desktop, rather than having content available when I’m away from my PC.
I suppose in the end it’s a business decision for you as to how you want to strike a balance between a mobile consumer solution and a power tool for intensive research with very large databases that are mainly worked on in the PC.
Just my 2 cents, while also trying to represent “power users’ interests 🙂
Hi Dr Andus, There is no plan to intentionally dumb Surfulater down, however considerable effort is going into simplifying and streamlining the user interface. See my earlier articles. Conversely it may well end up being even better for “power users”.
It will and is working on Tablets as well as Desktops, but is unlikely to be squeezed down to mobile phone style screens.
-Neville
Thanks Neville, that’s reassuring. Could there be perhaps a limited capture tool at least for the smartphone format? I’d think the smartphone+tablet+desktop combo for note-taking systems is becoming increasingly common. In fact I’m more likely to use my smartphone (OK, iPod Touch) for note-taking on the go than my tablet (iPad), considering the former is guaranteed to be with me 100% of the time (unless I’m in the swimming pool :).
Perhaps with the ability to create new templates, a user could create a template that is suitable for mobile phone style screens.
P.S. By “capture tool” I mean at least the capturing of free notes, not necessarily of websites, though that would be nice as well. Although having access to at least a text version of the cloud database would still be important.
Happy to take this on board.
-Neville
Hi Neville, it was good to read your latest thoughts on the issue; I read them with great interest. While I made my feelings about data storage in the cloud very clear in my last reply, I don’t have an issue with running a personal/private cloud. I just don’t want my data uploaded and stored on somebody else’s server. I do a lot of research on the Internet for business purposes and I collect information from many different sources to be verified, filtered and consolidated. And much of that research gets stored, sorted, tagged, and collated in Surfulater. For reasons I don’t care to go into here, I don’t want a copy of my database outside my exclusive control. It’s not that I’m doing anything illegal or anything like that, it’s that the data often becomes a sort of blueprint for my business projects.
That being said, I like the benefits that a cloud provides and you eloquently stated them above. In your “ideal world” solution you spoke of Surfulater running both locally and in the cloud with subsequent syncing. I would be open to that if (uh, I mean “IF”) I could specify whether a database was to be stored locally only or if it could be synced with the cloud. In that way data that I wanted to hold close to the vest never gets uploaded to a third party server, and I don’t have to worry about it.
I also found the SFF computer solution interesting. Another device worth considering might be the Tonido Plug (http://www.tonido.com). I think I read that they have an API so maybe you could get Surfulater to “talk” with it and use its cloud. It also has functionality well beyond just syncing data (i.e., NAS, music server, etc.), and it is very affordable at only $119 USD. (No, I’m not an affiliate or in any way related to the product or developer at this point in time.)
At the end of the day my main concerns are privacy and security. As for Surfulater, you’ve already convinced me that you can develop a very useful piece of software. And your concern for both camps of loyal users tells me that you’re the developer to support. I like using Surfulater and look forward to the next version…I just hope I can use it once it gets here.
Hi John, thanks the reply. One big thing that has changed in Surfulater Next-Gen is that their is only one actual (physical) database for each user, unlike the current Surfulater where you can have many.
However that database can have as many individual Knowledge Bases as you want, so in actual use they are quite similar. I only mention this because I’m interested to know whether you would be satisfied to differentiate between private local content versus potentially public cloud content at the Knowledge Base level, or is that too course grained?
I am well aware of Tonido plug, but haven’t looked at it in some time. It was quite expensive here in Australia although it appears that we can buy direct now and the $US119 price is far more attractive. That said Surfulater Next-Gen may struggle to run on such a device. It is definitely worth consideration though.
-Neville
Hi Neville, I understand the difference between the current version I use now and the Next-Gen version, and having multiple KBs within a single database. Given the inevitable direction things are moving in, as long as there was a fail-safe way to keep “private local content” private and out of a potentially public cloud environment then I would be fine with that. Not everything I store in Surfulater is private. For example, I also store how-to information for Windows and Linux issues and procedures. Those types of things could be stored in the cloud without keeping me up at night.
But let me ask you a question. While it is (unfortunately) true that cloud-based computing and storage are the new buzz topics and trend, they often go hand in hand with a recurring subscription fees, is that what users of the Next-Gen version can expect?
Typically server storage space and bandwidth has to be paid for, which is the downside of moving data off the local PC and into the cloud. And it’s par for the course that multiple parties try to get in on the action so they can stick their hands into somebody’s pocket. The ISP meters the user’s bandwidth waiting eagerly for them to exceed the monthly cap or limit, the AV companies want to sell their wares to protect the data and secure the connection, and the cloud service hosting provider wants their share on a monthly or annual basis for use and storage. And all this occurs after the user has paid the developer for the use of the software. These are the things that make me question why anybody wants to move their data to the cloud. And I didn’t even mention privacy issues. The stark realities are that TOS agreements tend to be mostly one-sided (in the provider’s favor), servers get hacked and information gets stolen, government agencies have issued subpoenas for user data while service providers comply, and litigious corporations regularly troll the Internet for anything that looks like copyright and/or IP infringement. And so many people want to blindly offer up their data to a cloud repository? I don’t get it.
That’s not to say that any of the aforementioned things will actually happen to a Surfulater Next-Gen user, but it is the direction the world is going in. So everybody should think very carefully about what they store in the cloud. (Wow, I can’t believe I just wrote that in a time where we have a Facebook generation that will indiscriminately post their personal information and wild escapades on a public “social” forum.) Okay, I’m stepping down from the pulpit now.
Hi John,
It is too early to say how pricing will work, however I would expect the cloud version to be subscription based and the locally installed version to be priced more like conventional desktop software with a one-off purchase and paid upgrades, but don’t hold me to any of that. 🙂
I personally think there are more upsides to moving *certain* data to the cloud than downsides. There was a time, not so long ago where I didn’t feel this was the case. Now I am gradually moving more of what makes sense to the cloud, to free me from the shackles of my desktop PC(s) and having to forever keep them in sync.
-Neville
Hi Neville,
Having had a few weeks to ponder the issue, and having just read Kate’s 12/10 post, I am rethinking my position on the single database. Even if you are able to securely segregate “private” from “public” data, any synchronization will most likely take place on the cloud server. That means that my entire database would have to be uploaded to the Internet at some point if I wanted to take advantage of the sync feature. Doing so completely undermines my desire to NEVER upload my private data to the Internet. So is it too late to program the new version to use one database locally and another database for uploading and synchronization? I don’t know why I didn’t think of that earlier — thank you Kate.
Also, I think Kate’s idea of using Surfulater as an interactive publishing platform with role-based accounts (e.g., read only, read/write, admin) is simply brilliant. I too would be able to utilize Surfulater to a greater extent with such a feature.
Hi John,
Synchronization to the hosted cloud can easily be done at the individual Knowledge Base level vs the entire Database/All KB’s. So any KB’s you didn’t want synchronized would only ever have their content on your local PC.
It could even go lower to a Tag’s level where only articles Tagged ‘Sync to cloud’ would be synced, however I think this is probably going too far.
Also see my reply to Kate’s post re. Databases. The roles Kate brought up need more considered pondering.
-Neville
Count me as another who does not favour cloud-exclusivity at all. Not one bit. I’ve been using Surfulater since its early days, and every data thing of importance to me is in it: journal, contacts, household expenses, farm planning and records, market recipes, urban-ag research, 11thC research, tech tips, consulting projects, writing. I am not always online, e.g., at the farm, at the camp — and I need my information available when I am not online. A personal cloud or a thumb drive sounds fine — what I need & want is no dependencies (other than electricity) beyond my computer system, and no security topic to think about at all.
That said, right away I see several ways I would use Surfulater in the cloud for shared information that is automagically updated by a number of invited users. Surfulater seems to me a terrific entrant in that field — it is so easy to use that my communities would have no trouble … it would become instantly essential to me in that role, and I can’t imagine why I haven’t thought of these applications for it before!
But — that data is quite other than the information I currently store; and I would not necessarily need synchronization with local database.
I am sorry to read there will be only one database in future. I have relied on being able to store contacts and journal in a personal folder, farm records and market recipes in the farm folder, etc. — and back them up separately. No one DB is very large. This is one of the many aspects that makes the app feel very “light” — not burdensome — to use.
I think there are three roles for Surfulater — research aid/record keeper, publishing tool, and (coming now) collaboration tool. Both the feature set and the data one would store overlap among the roles, but they are not contiguous.
Hi Kate,
Thanks for informative reply. Next-gen Surfulater does move it’s use cases well beyond the personal knowledge base app we have now. I’ll have to ponder on this and I guess to a large degree see what happens when folks eventually get to play with it.
As for the single database I appreciate the backup issue, however the advantages it delivers outweigh this. You will still be able to have as many knowledge bases as you want, but now you’ll be able to search across kb’s which has been much asked for. You can also have an article in many kb’s, although I’m undecided whether to enable this. It is also much quicker as all kb’s are in essence always open. And it simplifies the entire design.
Having said that a database is tied to a user (at present their e-mail address) so there is nothing stopping you from having multiple databases if that’s what you want. In the local personal cloud version these wouldn’t cost you anything, but in the hosted cloud version you would likely be paying for each database.
Thanks again for contributing to the discussion.
-Neville
Some more perspectives on this.
1. I work for a research organisation in Canberra, and use SUL to collect information about a wide variety of topics. The use-case of offline access is absolutely critical. Wi-fi is NOT ubiquitous, and cellular coverage is often flakey at best. I work next to Black Mountain – with all its repeaters and mobile phone antennae- and can usually get only one “bar” of coverage via Optus. Further, there is limited organisational support for wireless internet. Online access is appallingly slow. If SUL goes cloud-only, I will have to look elsewhere.
2. The folder/tagged/chronological views are all useful and I’d hate to lose any of them. The chronological view gives a different context and findability to articles: if you can’t remember what an article was called, or how you tagged it, you might remember when you added it.
3. John’s comment on subscriptions is quite valid: I don’t mind paying for applications and their upgrades, but having to pay for access to my data by annual subscription? SUL NG would need to be pretty compelling.
Of course one way to solve this is architecturally, whereby you can have a local “cloud” (database) or something remotely-hosted.
I suppose we will all be a little more reassured once we can play with a SUL NG beta, and be able to put it through its paces in real-life situations.
Hi Nick, thanks for your feedback. The locally installed personal cloud I’ve discussed previously seems like the way you will want to go.
So far I haven’t implemented the Chronological view as such, instead articles are displayed in Chronological order. Other sort orders may well be added.
-Neville
Hi Neville,
Do you have a rough idea of when this next-gen Surfulator is likely to be available?
If I got the current version in the meantime, will the info I generate using it be compatible with the new version? And, finally, presuming it is, would it cost much to change over?
Thanks and regards.
Ingolf
Hi Ingolf,
I don’t have a specific time frame for release at present. I have recently moved the Alpha version to a new remote (overseas) Web server for testing, which has and is helping to iron out various latency and similar issues. I continue to make good progress, but there is is still a ways to go.
I do plan to enable content to be transferred from Surfulater, to the extent possible taking into account the differing capabilities and feature sets of the two applications.
At present the I’m looking at two versions as discussed in this post. A hosted version with an annual subscription and a locally installed version which would likely be a one of payment and then the option to purchase upgrades.
-Neville
Much appreciated.
While I really look forward to being able to access my SUL database from all of my computers (to me the only really compelling feature of Evernote), I’m not thrilled with the idea of a web-based app. My experience is that they are not as responsive nor as user-friendly as PC-based apps, and tend to suffer when browser versions change. I prefer the Evernote model of PC-based app and locally stored, but cloud-synchronized data.
I agree that having a cloud + local component is the best way to go. I’ve been researching information for an upcoming trip and have saved many web pages in Surfulater. However I am not bringing my notebook along and am having a heck of a time trying to figure out how to get the Surfulater info into my iPad and Android tablet for portable viewing.
-Neville
PS. In reviewing various related communications I noticed a request for a Portable version of Surfulater (ie. one that runs on a USB stick.) Isn’t using Surfulater as a Web application in the Cloud the ultimate portable version.”
Actually, regarding your reference to a Raspberry PI model – “Surfulater-on-a-Stick” is the ultimate! (that is, encrypted program & data both resident on a USB stick)
@xman Clibu is Surfulater in the cloud and you’ll also see a locally installable version of Clibu and quite possibly Clibu on a stick. Lots of great stuff happening here, so stay tuned. And if you aren’t using Clibu yet you should be. 🙂