Portal Home > Knowledgebase > Articles Database > Trying to understand the cloud
Trying to understand the cloud
Posted by TheJoker, 03-01-2011, 12:24 PM |
I thought the cloud was the same "VPS" that is hosted on geographically separated servers offering redundancy. But now my impression is that if I purchase 3 nodes on a cloud, I can either lump them together as one big node and run all my services from it, or use them for 3 specific purposes. What I can't wrap my head around is how is this any different than having 3 small VPS or 1 big VPS? What makes it the cloud?
Last edited by TheJoker; 03-01-2011 at 12:29 PM.
|
Posted by tchen, 03-01-2011, 12:32 PM |
The clouds using the node-resource allocators should let you scale dynamically without reinstalling anything. The VPS are fixed so if you wanted to switch from 3 small VPS to 1 large one, you'd have to reinstall and migrate everything.
If you're not going to be changing allocations, then it doesn't make much sense to go with the node-base system, IMHO.
|
Posted by ghMike, 03-01-2011, 06:09 PM |
Being able to scale up and down is the big difference in cloud hosting, you usually have some sort of simple control panel as well. There are virtual servers hosted on cloud infrastructure, but they act more like VPS, but with built in fail over. The real issue is defining cloud technology.
|
Posted by cphaynes, 03-01-2011, 11:15 PM |
Couldn't have said it better myself. The funny thing is, most people think their "cloud" server is some magical technology monster with feet in one city and hands in another. In reality, most "cloud" providers today just offer VPS type hosting that can scale HDD/Memory on whatever host it's running on - on demand.
To me, Cloud = SuperVPS.
Of course, another side to all of this is - say you use Google Docs for school/work, etc. Well, those documents are in the "cloud" because they aren't on a single physical machine somewhere. They are saved, copied, and accessed online. If you type a document in Google Docs in San Francisco, Google replicates that document instantly to all their data centers. So, when you arrive in Atlanta and need to access that file, you pull it from their ATL data center - because Google knows via Geo where you are.
In a nutshell, ghMike said it best. Defining cloud tech is the real issue. It's more of a marketing term now (it seems) than a technology you can put your thumb on.
|
Posted by musicman153, 03-03-2011, 05:19 PM |
I have trying to figure this out myself for some time now. From what I able (in my limited capacity) to understand is the term "Cloud" just refers to storing information on a server as opposed to your computer? I would think this is what we have been doing on the "Internet" since the creation of email. If I want to save a document I can send it from my hotmail to my yahoo email and it is accessible from anywhere.
Why would I need to access a server specifically in Atlanta when if I'm in Atlanta I probably still have access to the server in San Fransisco? And why would I need to rent a VPS at all as there are plenty of places already out there for file storage most of which are free?
Still trying to understand
|
Posted by phpcoder, 03-03-2011, 06:49 PM |
I'm not sure I follow your logic, would you care to elaborate? Of course you're going to need to migrate everything if you're trying to combine three systems into one.
Again, it depends whom you ask that question to. For instance, Microsoft wants everyone to "move to the cloud" (excuse me if I've missed their exact phrase) which in the grand scheme of things means store your files on their servers rather than your desktop. That isn't cloud computing to me.
To me, in simplistic terms, cloud computing (for websites) would allow a website to be hosted in an environment that allows complete auto-scaling (ie. 20k visitors one day and 10 million the next) with no noticeable performance impact. Additionally, I would say cloud computing SHOULD mean that ANY hardware failure will not impact the website. Unfortunately, that doesn't exist today at a price that 99% of people can afford.
Even so... you then have software failure to consider.
But, as we've said, everyone has their own interpretation of what "the cloud" actually is.
Here is a great video clip from Larry Ellison poking fun about cloud computing which I think you may like!
http://www.youtube.com/watch?v=8UYa6gQC14o
Last edited by phpcoder; 03-03-2011 at 07:02 PM.
|
Posted by tchen, 03-03-2011, 07:42 PM |
I think I misunderstood what the OP was asking. I thought he was asking about how hot-resizing of node allocated systems compared to normal VPSs.
Now looking at his post again, I'll have to answer that there is no difference other than automated monitoring and failover at a lower level than you could achieve yourself.
|
Posted by musicman153, 03-03-2011, 07:46 PM |
Thanks for the link I love it
So basically it has nothing to do with "Nodes" or VPS but just a new way of marketing new (and old) technology?I think I'm starting to understand
Thank you
|
Posted by tchen, 03-03-2011, 07:56 PM |
Amen to that
|
Posted by wartungsfenster, 03-03-2011, 08:01 PM |
I'm still working to get to the difference between public grids (like i.e. planetlab) and clouds, except of course the billing.
No just kidding. Cloud is timeshare computing of the 60s with a faster network cable. Any people that fall for the marketing trap and put both their servers (EC2) and storage (S3) at one provider are going to be ripped off and those who buy IaaS (anyone remember the term "application service provider"?) are totally asking to get their last dollar taken.
You put your data into a proprietary app (that means you don't control it's structure any more and can't easily put it with another provider/app)
You put your data onto a cloud storage (which is rarely crypted, never with YOUR key) that is not under your control and written directly by the app, so you can't just pull it to your own server
You put your compute ressources with some online cloud vendor...
Grats, your computer is now a dumb terminal and in 3 years from now you won't have the slightest idea where your stuff is.
(Later on things will get better and they'll let you export it
At some point analysts will notice that shops with their own IT run circles around the ones with standardized call-center-backed IaaS applications. Then we spin back and
round
and
round
it goes...
More seriously I think the best way is to set up "cloudbursting" so that you'll be able to use on-demand compute power, but by standard you run your own systems (which is a lot cheaper, too) and also have your primary data copy on your own boxes.
Offering cloud servics is my main goal, but I think the turn to a real commodity and quality product will only come when the customers unlock the data access via some crypto protocol and there is a 100% standardized protocal of the oject storage.
Until then we might serve small startups, but I think cloud services would be far more interesting for the fortune 100 who all run their own large-scale datacenters that idle 95% of the time. They are the ones that have real demand for peak processing power, but right now they couldn't possibly use cloud ressources because noone bothers to think about data safety standards.
|
Posted by JSCL, 03-03-2011, 08:10 PM |
Not really. It depends how it's done - many of us describe the basic cloud as 'vps on SAN' but a real cloudnis raking our typical VPS to the next level - creating a stable and secure environment in the Cloud. Don't be fooled by many providers claiming cloud when actually it is just the basic 'vps on SAN' because Cloud is actually much more than that and it's more than just the ability to scale up and down within a cluster. It's about redundancy and the platform/software backing the cloud also.
Redundancy could mean anything but theres several levels - firstly of course the SAN's and their redundancy and are they locally redundant/failover or geographically placed elsewhere which covers more protection based on anything that could possibly happen to a facility. Then you have redundancy with similar situations but n the actual hardware providing the other resources - the systems in the cluster.
That's a really basic way to look at it but sure, cloud is old technology but it's important in modern life and it's where we're heading. Most Google applications are cloud based and all your files are stored in the cloud. Many peoplenuse Google apps as a comparison of Cloud vs local. But it's not just a marketing ploy - it's making better use if existing technology and on a larger scale because it's better than local. Cloud delivers so much more than a standalone dedicated system for your website or whatever else you use your servers for. If you are considering between 'cloud' or a normal vps - go cloud. Better performance, more control and a safer way to store your data and host your website. RackSpace shares didn't just boom because of Cloud because they play a marketing ploy well - it's because businesses appreciate the importance of their data and websites and would rather be in e cloud.
|
Posted by wartungsfenster, 03-03-2011, 08:47 PM |
Even more since most people will call it a SAN when they merely deployed a few low end iSCSI arrays and a Dell Gigabit switch. :/
Would you agree a cloud application should be able to scale out (not saying it should squale linearly), but it should definitely be able to run on 1, 2, 5 or 100 systems?
And if one system dies out only the transactions on that one should be affected?
Like, in the easiest example a CDN or like in past days a larger bunch of apache VPS with mod_backhand, or like Oracle RAC in the ideal case.
Because I think this is where a cloud application would differ from having a bunch of VPS with clustering and a SAN backend.
|
Posted by tchen, 03-03-2011, 08:53 PM |
@JSCL
So, isn't that just saying again that it was a rebranding of the redundancy operations that large dedicated IT personnel were doing ten years ago? I will give credit to the latest cloud-trends in that it has exposed that expertise and driven down the procurement costs such that it is practically available to a wider audience. That's why Rackspace shares went up, not some fundamental shift in the reliability or any change at their DCs.
Last thing though, I'd be careful about comparing the recent wave of cloud providers to Google. Outside of the web frontends, the core of their system (what makes google, Google) is principled on a _very_ different architecture from a hardware-centric cloud. You're getting into the realm of scalable computing - the actual technological and academic field focused on this - not some marketing hoopla.
But I'm just a curmudgeony technologist and not a marketer. Take my ramblings for what they are.
|
Posted by musicman153, 03-03-2011, 09:05 PM |
Interesting (or confusing),
I suppose that this may have a huge amount of relevance for mobile apps as you have a limited amount of storage space but I do not see myself designing a website from a phone any time in the near future. (of course you never know)
I can appreciate the latest technology as those in this industry would be shooting ourselves in our (cloud based) foot not to keep up with it.
Again though we've been storing information on outside servers since the beginning. I'm still not seeing how this is different. Perhaps I'm having a hard time with the techno terms. can you simplify it. Just pretend you talking to a second grader lol.
Thanks
|
Posted by phpcoder, 03-03-2011, 09:30 PM |
I assume you mean something along the lines of: "cloud" (now) vs. server clusters (< 10 years ago)?
|
Posted by tchen, 03-03-2011, 10:01 PM |
That kinda implies they're using old technology which to be fair, isn't the case. I find it easier to talk about them as parallel tracks.
On one hand, we have those people who have been managing the server clusters since 10 years ago, and are still doing it now.
And on the other hand, we have cloud providers who are doing the exact same thing, although they're selling their expertise and setup externally to end consumers instead of being employed by any specific company.
Both sides are pulling forward the best practices of 10+ years of experience.
|
Posted by tchen, 03-03-2011, 10:26 PM |
It depends on how nebulous you want the term "Cloud" to mean, and partly your background. Ill admit, I think people like yourself and whove had more exposure to computers than just their personal standalone desktop are harder to sell Cloud to.
The modus operandi is still the same as when we had mainframes. The difference is that over the years, the computer youre connecting with have gotten faster and smaller, the network hardware has gotten smarter, and the storage solutions have gotten more robust, and the IT personnel have gotten more experienced. All this happened before cloud was a glimmer in someones eye.
That progress is still happening now and Im sure whatever technologies and techniques we have in the scalable computing architectures will come down into the cloud one day or whatever it may be called then - for everyone to enjoy.
What kind of stuff? Processing of large sets of seemingly unrelated and useless data is one. Google knows what you might be eating, and Facebook knows what movie youll be going to see next. The architectures they have in their datacenters are beyond what you see being sold to outsiders as the cloud. As wartungsfenster alludes to, these architectures work on a single problem, at the scale beyond 100s, well into the thousands of computers and require a specific network setup and different reliability assumptions.
But arent these still considered clouds? They dont think of it that way internally. To them, its still a network engineer and her datacenter. Someone might want to pin the name on it and call it thusly, but thats just redefining and chasing the current state of the industry and makes for a poor definition.
|
Posted by JSCL, 03-04-2011, 06:38 AM |
Sure, 'cloud' in itself is old technology in the sense that corporations like major banks for example have been using clouds for many years. The only reason it's now generating a buzz is because only now are we seeing it is affordable t bring to the consumer markets like WHT at the similar price to a normal dedicated server. It allows IaaS providers do deliver to consumers and also to corporations who didnt necessarily want to run and operate their own clouds. But the fact is, cloud has been in our lives for quite some time - we just never really thoroughly thought about it. Your online banking, YouTube, google apps and google docs - all these are examples of cloud in use in our lives that normal internet users didn't even know what cloud was or how it affected them - this is how it does.
Clusters have been used even in the general consumer market for years, but the idea of a cloud with redundant SAN's and an easily scaled virtualized setup is kinda phase 2 of clustering I suppose you can say. Given the choice - a 200 dollar /month dedicated server with a single drive and no redundancy or a 250 dollar cloud environment of the same specs but completely redundant - which should you choose? Personally? I'd pay more for cloud.
|
Posted by tchen, 03-04-2011, 09:05 AM |
Call me crazy for wanting to use a cluster in a cluster comparison, but lets for the sake of argument say we ARE building a simple web cluster.
2 load balancers, 2 web servers, 2 mysql databases. 200x6=$1200
Which gives a reliability as good as your DC's network.
On the 'cloud' environment with SAN, you still need to build the same with the virtualized servers. 250x6=$1500.
What does the extra $300 provide you? In truth, its insurance coverage for one of the most infrequent of failure risk scenarios - the dead brick server. But at that point, I might as well provision myself another hot spare in the boring old cluster.
|
Posted by brentpresley, 03-04-2011, 09:14 AM |
I think your numbers are a bit off. If you do not require an external SAN, you can setup that cluster for a fraction of the cost, and have uptime that is as good as your DC's network connections (which if you are really going for good, you get dual, independent feeds of both bandwidth and power).
|
Posted by JSCL, 03-05-2011, 08:41 AM |
The end part is part of Cloud, sure. But Cloud is comparatively new? Cloud is 'old' technology/ideas and has existed for many many years.
|
Posted by tchen, 03-05-2011, 01:01 PM |
Are you referring to the drive mirroring? Or am I missing something else obvious.
|
Posted by brentpresley, 03-05-2011, 01:07 PM |
I was referring to the entire hardware setup. You can deploy 2 machines with AppLogic in a private grid and accomplish everything you listed (2 load balancers, 2 web servers, 2 mysql DBs). You run them off the local IP SAN, so no external SAN is needed.
Overall cost is significantly lower.
You have full High Availability, and as long as your DC has dual independent power feeds and dual independent fiber feeds you are about as good as you can get from just one DC. If you need more resources, just add a node to your grid, one by one, till you get what you desire.
|
Posted by tchen, 03-06-2011, 01:19 AM |
The above is HA with the exception of your DB locking backup. There's a reason why the DB is a 2-block; its not just for failover.
I will concur that if you were going to treat the second web-server as an unutilized hotspare, then you can get rid of it in the AppLogic setup. Likewise, I could get rid of it in the cluster. And to be fair, let's relax the HA requirement then to add the 3 minute detection and 4 minute startup time for a node failure. If you're willing to give me 3 more minutes on top of that, I can drop the load balancers too and just use ip failover. I'd still need one spare server, plus 2 minutes for auto configuration.
There's no denying there's a sweet point where if you prune enough of the requirements, AppLogic can meet the demands better and easier. The best that the non-AppLogic cluster can get down to, while still retaining the barest resemblance to HA, is 3 machines. Two if you're willing to colocate the database and webserver vs the one for AppLogic in that scenario.
Going back up to the other end of HA, you just can't rely on the base level cloud to provide you the response time and reconfiguration required. You start adding more machines. Your network spans out geographically. Your application layer starts to handle your own monitoring and reconfiguration via heartbeat, pacemaker, monit or whatever your favorite tools are.
Short of virtualization, both the cluster and the app on a cloud starts looking exactly the same.
|
Posted by brentpresley, 03-06-2011, 06:12 PM |
Agreed. I am assuming that 2nd node is used 100% as a hot spare node.
Sure, can you design a better setup with more hardware and nodes? Absolutely. But for bare minimum setup to provide HA, you can do it that way. And we have setup companies on VPDCs like this with the option to expand nodes in the future as their business grows.
|
Posted by MattS, 03-06-2011, 07:14 PM |
Cloud hosting to me would be like taking multiple servers and gathering all of the resources I need/ want (nodes as some companies call it). In a cluster you would dedicate each machine to a different task, whereas a cloud they would all do whatever needs it.
Not sure if there are any cloud clusters yet, but give it time and those will come to be as well.
Combine the best of both worlds, a cluster for the stuff you want and a cloud to handle everything else
|
Posted by wulaishiwo, 03-06-2011, 10:25 PM |
In short,the cloud is more convenience than VPS,someday it must replace VPS.
|
Posted by tchen, 03-06-2011, 11:31 PM |
I think that's probably the most cost effective. I have EC2 instances on standby (but otherwise off).
|
Posted by jNive, 03-06-2011, 11:55 PM |
The ideal occurrence of the 'cloud' is a computing resource not tied or dependent on any specific or single physical location/object. All resources within the cloud should be simultaneously accessible from all external points, and internally via multiple/infinite paths- In an ideal world :-)
As technology improves, and infrastructure sizes increase - this is becoming more and more of a reality. These days the terms 'cloud' almost used in relation to any level of hosting without an exact 1:1 relationship with physical equipment (e.g. a VPS that can be moved on request from one server to another)
|
Posted by Michal parke, 03-07-2011, 06:36 AM |
The cloud hosting is the service of web hosting which is distributed form a set of connected servers. The phrase of cloud hosting is not exactly defined so far. In usually, however, a web hosting service can be think about cloud hosting when it is distributed from a totally disused, multi-server system, in which the assets are energetically scalable and often effective.
Working of Cloud Hosting
A cloud hosting service is generally distributed from a load-balanced group of server stage, while the informantion is save in a Storage area network.
Cloud hosting contributor determine compute cycles, a pointer which detetmine how much handing time applications need. This allow cloud hosting to be afford and billed as a section of a unlike service model than that used by other web hosting services.
|
Posted by Podtech_Innovation, 03-08-2011, 04:48 PM |
IaaS is absolutely NOT the same thing as the old ASP. SaaS is the latest equivalent to ASP. IaaS is infrastructure (hardware, network, power and in some cases an OS) - it has nothing to do with what you are talking about
|
Posted by Giraffian, 03-13-2011, 06:34 PM |
This is an interesting thread - yesterday I had someone trying to tell me that I was a fool to want a bunch of dedicated machines and that cloud would be perfect and cheaper.
My app is basically a metered fileserver, where every access of a file needs to be logged and paid for. We were going to do this with a heirarchical 'control' module that hands out 'blocks' of these accesses to children in the heirarchy, and the children report back when either they've run out of block and need more or when the block is no longer needed and they hand back the remainder. Of course, in the meantime, there needs to be counters kept on the child machines that are saved somewhere, so if a child machine does go down (assuming its not a catastrophic hardware failure that wipes out every level of a raided disk) it just means that that particular block won't be served for a little while until the child comes back up again and then things resume from where the count was before, which I can live with. Even having the master go down is only a problem if it stays down so long all the children run out of blocks to serve.
I'm extremely concerned that if I did this on the cloud, a child server would actually *lose* the counts it has stored, where if I did this with dedicated computers or even VPSs I keep the data I wrote when the child goes down. Cloud seems to me to be more of a pure replication-and-synchronising technology used to make a comparatively 'small' website viewable by a massive number of people at once.
Am I wrong? Because I was certainly made to feel like I was wasting my time thinking dedicated yesterday.
|
Posted by tchen, 03-13-2011, 07:38 PM |
Giraffian, your problem really seems more like an application level thing than something that either a cloud or dedi would affect. Granted, a cloud on a SAN would survive a tad better since the counter's survivability is slightly improved as it can account for non-raid hardware dropouts.
But have you considered moving the count storage and control into a clustered or replicated data store? There's a whole slew of fault-tolerant ones out there, from CouchDB, MongoDB, Redis, to Cassandra. The last one is probably the best for you as it can sustain quick blind-writes and is automatically distributed in a ring so it can sustain dropouts quite well.
This can be built on top of either cloud or dedi, or even VPS for that matter.
|
Posted by Giraffian, 03-13-2011, 08:39 PM |
Yeah, its an application thing, was going to run it as a service, probably as a httpd module. I don't mind which hosting solution as long as my counts stay absolutely accurate, since they can't be replicated - every machine has its own unique counter.
I just don't know how all these cloud instances handle data that is *written* to them when they shut down. If it is still there when a cloud instance restarts and comes back, that's great, but I can't find any information about this other than someone telling me "oh you can do that easy". Synchronising over the cloud doesn't seem too hard either, but I don't want to synchronise and duplicate.
Edit: the discussion was entirely about Amazon cloud, which is the one I don't understand, as the cloud instances just seem to pop out of nowhere and disappear again when they're not needed.
Last edited by Giraffian; 03-13-2011 at 08:45 PM.
|
Posted by tchen, 03-13-2011, 09:59 PM |
Amazon's cloud has two types of attached storage. First is the local ephemeral one which when the instance pops, disappears. This writes directly on the hardware node's HDD. Another one is the elastic block storage (EBS). Which simply put is similar to the SAN storage some other providers use. EBS volumes are persistent and will survive your instance going poof. It's also a lot slower.
I think when someone says 'synchronizing over the cloud is easy', they might be referring to one of the noSQL databases I mentioned. There's a lot of tech out there to selectively synchronize data and coordinate multiple nodes, whether they be cloud nodes, or dedis/vps.
And to be clear, when I said replicated, I meant the datastore, as a whole. The counts would still be tied to their individual instance via a key.
That said, EC2 is the worst place for a file server given the traffic costs.
|
Posted by Techy, 03-13-2011, 10:02 PM |
This is an interesting article about the cloud, you should check it out. see here.
|
Posted by tchen, 03-13-2011, 10:27 PM |
I'll point out again that the article was a poorly written sensationalist piece that tries desperately to make it an either-or scenario. It's made to sound like the business talked about like Netflix and Etsy are hosting directly out of the EC2 - which they are not.
|
Posted by Techy, 03-13-2011, 10:28 PM |
It's a very interesting article though, IMO. It may not have 100% of the facts but it's definitely interesting.
|
Posted by Giraffian, 03-14-2011, 01:45 AM |
Haha yeah I did notice that, the advantage of just a plain ol dedicated is they come with a LOT of included bandwidth.
Just looking at the standard couple of servers and loadbalancers at liquidweb at the moment. Cloud stuff is a bit new and funky. I must be getting old ...
|
Posted by alex18, 03-14-2011, 04:24 AM |
Hi,
sorry for just jumping into this topic but I have a question that is related to this topic. Is it right, that there is no Hypervisor that allows a cloud-server to run on multiple physical servers ? Because it looks like when you order the highest available spec. for a cloud, you get a high-end physical server for your own.
|
Posted by brentpresley, 03-14-2011, 08:55 AM |
This is correct. There is no interconnect between servers that is fast enough to allow for data to stream at the rate that would be necessary for a hypervisor to coordinate multiple computational threads on multiple nodes within a cluster/grid/cloud.
An alternative to this is to break things up into descrite chunks than can be run on specific nodes within a cluster/grid/cloud. LAMP is one example whereby you run a webserver on one node, a database on another, both controlled by a load balancer on a different node. There are other examples.
|
Posted by alex18, 03-15-2011, 04:12 AM |
Thanks for your detailed answer ! Did you read about the ceph file system ? http://ceph.newdream.net/ it looks like this is a real cloud file system that acts as a small CDN too.
|
Posted by brentpresley, 03-15-2011, 09:51 AM |
Interesting read. Distributed file systems are definitely the wave of the future, but which one becomes the standard is still a good ways off I suspect.
|
Posted by wartungsfenster, 03-15-2011, 02:11 PM |
I'm playing with Ceph for around a year now - it has very very elegant and robust design, which sets it apart from most of the other "new" distributed filesystems.
In my tests it's been more stable and a lot easier to configure than for example GlusterFS, which is assumed to be "production ready".
Still there's a lot of warning messages about it's experimental status... Once these are gone, my bets are on Ceph.
Reasons:
- Clear design
- Scales well in Throughput and IOPS (even metadata operations)
- Robustness (rebalancing and all that stuff actually *works*)
- It works fast as a real (posix) filesystem, unlike some others that you can only access via HTTP...
|
Posted by TYQuan, 03-16-2011, 12:14 AM |
i've got a question:
under what circumstance would cloud be a more beneficial option?
|
Posted by JasonD10, 03-18-2011, 08:57 AM |
What? The Larry Ellison video? That is so.... 2009. Oh, and that's the SaaS model he talks about not the IaaS model. And on a final note since that ridiculous public rant Oracle has completely embraced Cloud.
|
Posted by dotHostel, 03-18-2011, 09:20 AM |
To be fair, for many years by now the Internet is usually / popularly pictured as a cloud.
Agree 100%
Last edited by dotHostel; 03-18-2011 at 09:34 AM.
|
Add to Favourites Print this Article
Also Read
403 Forbidden (Views: 1012)