Welcome! Log In Create A New Profile

Advanced

Building a server, any words of wisdom?

Posted by Cab Treadway 
For a while now I've been toying with the idea of using a somewhat leaner (physically, at least) computer as our "main" machine and building a file server to handle the storage. We/I have been shooting a lot of photos, and at 12-15MB per shot, the volume is building pretty steadily. I currently have 250GB, 400GB, and 2x 500GB HDDs installed. The twin 500GB are my newest, and they are purely for photos. I import files to the first as my main storage, and back up copies to the second. I guess it's not officially a RAID setup, but that is basically my intent. I did some calculations and figured that with 500GB, that would probably last us about 3 years of photo shooting, unless I ramp up my volume even more. I elected not to go with TB or larger drives because if one does fail, either now or when archived in the future, I'd rather lose fewer files - not putting all my eggs in one basket, so to speak. When these drives get full, I'll get a couple of new ones and continue the process.

At the same time, the WinXP Pro machine I built up a few years back has been giving us an increasing amount of grief. A while back it crashed to the point where I couldn't boot it even in Safe Mode, and after a (bad) diagnosis I sent in the mobo for repair, but they found nothing wrong with it. I was able to restore the system by running the recovery console, but it's been intermittent at best since then. It would reboot all by itself for no reason. It would crash and/or hang if just let sit idle for any length of time. And finally, yesterday it crashed to the point where I couldn't even run the recovery console after booting from the CD. This experiment of building my PC has also taught me that while I may be competent enough to build it, I'm not really interested in learning what I need to to be the main IT support guy. I just don't know enough about how the system works to be able to troubleshoot when there are issues. I'm wondering if my old C: drive is failing and/or is somehow corrupted. But that's neither here nor there.

So.....

I went out and bought myself a new machine. It's nothing fancy, an eMachines desktop with Win7 Home Premium (64-bit), 4 GB RAM, and a dual-core AMD processor. Our home use of the PC is pretty much for going online and editing photos in Photoshop and Lightroom. It will be more than sufficient for such usage. I installed PS & LR last night, and I think they're happier with the 64-bit OS than they were before.

My plan with the old system is now to nuke the OS and give the PC a new life as a file server. I'd rather have a tower in the basement with all the big HDDs storing the files, and just use the new machine to import and work on photos, then keep them stored on the server. Lightroom is great at managing catalogs of files, so I think this will be perfect for me. I can import the files from the memory cards, save them over the network to the server, then process them on the new machine (and it doesn't care where the files actually live).

I am thinking that setting up the server running Linux would be the best way to go. It's supposedly very stable, gives a lot of control, is well supported through its pool of users/developers, and is free (I like that part!). I believe I already have all the hardware that I need to do this, and from what I've been reading, it seems as if all (yeah right, as if it'll be that easy) I need to do is install Linux, then download the Samba package to interface with Windows systems, and I'm basically good to go. Please enlighten me about what else I'll need to do, why my optimism is unwarranted, how I'm likely to fail, lose all my data, and set the house on fire in the process. A good dose of reality would be welcome.

Basically, my cadre of hardware consists of an Asus mobo, dual-core AMD processor, 4 GB RAM (I might steal some of that and add it to the new machine, if that's possible - since I've read that servers don't need a ton of RAM), and the aforementioned HDDs. I'm leery of using my old C: drive (the 250GB HDD) as the system drive, in case it is failing. How can I test its integrity? I was able to install it in my external HDD enclosure and access its contents on the new machine last night, so maybe it's not corrupt, just bloated and somehow startup files/registries are getting changed so it crashes? Whatever... I'm not terribly concerned with troubleshooting why the system has been crashing. That ship has sailed. I just don't want to install my new server system on that drive if it is corrupt or becoming corrupt. I'll partition some of the 400GB drive as the new system space, or go out and buy a new small HDD (50GB or so) to just house the new system.

Anyway, a sanity check from someone who's done this before would be much appreciated. Step by step instructions (download X, install Y, plug it in to the wall) would be even more appreciated.

Are there glaring problems with my plan? Should I just head down to the local playground and get one of the technically savvy 12 year-olds to come do this for me? Should I give up, drink beer and buy a Playstation?

Thanks!

Cab
1990 325i(s)
2004 325XiT
Your idea of building a Linux server is absolutely the right way to go thumbs up

I think you will be pleasantly surprised at how easy it actually installs and interfaces with your Windows systems.

My HP systems both went the way yours has :X Fortunately, I had a SATA/USB external drive that I was able to use to look at the 'damaged' disk on another computer. All my files were fine; it's just the Windows stuff that gets clobbered, namely the NT core. That's why you can't even boot in safe mode.

I've been running Linux(Ubuntu) now for almost a year with no problems other than a botched upgrade which was entirely my fault. Unlike recovering from a Windows disaster, it took me less than 1/2 hour to reinstall everything; including my favourite apps.
This was partly due to the tremendous amount of help available from the various Linux forums and help blogs.
New server, well and good, don't throw out the old one. Had similar issues with my machine at work. Turned out the power supply was going. Intermiitanat voltage was wreaking havoc. On my home machine, tuned out my ATI video card was wreaking havoc. Found many people on the web with the exact ATI symptoms, no fix. no one knows what causes it. Got a new video card, reloaded, all is find. No more AT for this kid.

alan
The new machine isn't a server, just a small form factor PC that is replacing our old WinXP machine. I don't think I have anything wrong with the hardware on the old machine, and I'm definitely not getting rid of it. I did wonder if perhaps I had overloaded my power supply when I added in the two new HDDs, but the BIOS voltage monitors seem to be fine. Anyway, I'm going to give this Ubuntu thing a try.

Cab
1990 325i(s)
2004 325XiT
Sounds good, Archie. I'm downloading Ubuntu Server right now, and I'll see if I can find time tonight to install it. Reading the instructions online makes it seem ridiculously easy, I am finding it hard to believe that it could possibly be as easy as it seems. If it is, I'll be a super happy camper.

Yes, when I checked out the c: drive with the external enclosure, all the data looked fine, so that's nice. I'll probably just leave that alone when it all gets running again, and just not use the system portion of that drive.

Cab
1990 325i(s)
2004 325XiT
Well, in my case, the bios stuff looked fine. For giggles, I took out the power supply and opened it up. There was some ugly black goo oozing out of the capacitors which was my first sign all was not right. If you have a spare machine laying around, it is a pretty easy swap out.

alan
Quote
Cab Treadway
The new machine isn't a server, just a small form factor PC that is replacing our old WinXP machine. I don't think I have anything wrong with the hardware on the old machine, and I'm definitely not getting rid of it. I did wonder if perhaps I had overloaded my power supply when I added in the two new HDDs, but the BIOS voltage monitors seem to be fine. Anyway, I'm going to give this Ubuntu thing a try.

Hi Cab!

Check the power supply, it may be failing causing the system to stall or reboot randomly, and ending up damaging the OS files.
If that happened, it will happen to your new Linux server setup.
It happened to me, a new power supply cured it.
While at it, check the heat sink and fans, overheat can cause malfunctions and end up melting something.

Good luck!
The saga continues, AKA I just can't get a break...

Short version:

I ordered a new motherboard last night, should have it in a couple of days and have the server set up shortly thereafter.

Long version::

After reading up on installation packages, considering my needs, etc, I decided that running Ubuntu Server with the Samba package was going to be perfect for me. Server is leaner than the full Ubuntu, and since I am not planning to use the machine other than as a HDD repository, I figured that was great. Less to go wrong, should just happily hum along in the basement unattended for months at a time, right? Samba lets Ubuntu interface with Windows systems on the same network, sounds perfect to me. So I downloaded the Ubuntu Server image, loaded up a CD and was off to the races. I intended to use my 250GB drive as the system, with the larger drives reserved for storage and backup. When I had connected the 250GB drive to my new Win7 machine, it found errors and went though the whole "fix-it" routine, so I thought that maybe the drive had started to go bad but now it was okay-ish. So I backed up everything from the 250GB drive onto the new system and then back in the old PC, I booted up from the install CD.

It asked me all the setup questions, and seemed to be on its way, but it would hang up on something. It was never the same thing, it hung on different steps at different times. It would partition the drive but would never complete the install. Keep in mind that I was doing this on a HDD that I thought may (or may not) have some defect. Anyway, it got to the point where it couldn't see the HDD, so I figured, "yep, this drive is hosed, might as well get a new one and start fresh."

Off I went to Best Buy, where the smallest SATA 3.5" drive I could find was 320GB, and for $5 more I got a 500GB drive. I only needed something like, oh, a 1GB drive since all I wanted to do was use it for the system, but retail stores don't really stock those, I guess. So I figured I might as well use the storage.

Back at home, even using the brand new HDD, the install got hung up, and then after having to do a hard shutdown and try again, it wouldn't recognize the drive. So it seemed that during the partitioning stage, it was doing something to the drive that, until the install is complete, makes the HDD look corrupt or not even there. Pissed off that I'd seemingly just damaged the brand new HDD, I put it in the external enclosure and went to disk management on the Win7 machine. I was able to find it, delete the partitions, and reformat. For grins, I tried the same thing with the 250GB drive that I thought was dead, and it worked, so I guess in the end I didn't need to buy the new HDD after all. Whatever. Now I have it.

(It's kind of annoying that the Ubuntu setup CD doesn't have a way to format drives. I'd think that would be a useful thing to include on the image, so you could clean the disk and start fresh, but I had to fumble around with my XP CD and then the Disk Management utility in Win7 to get the drives formatted, so I was doing a lot of juggling HDDs back and forth between machines. Where was I?...)

Interestingly, when I was partitioning the HDD during the Ubuntu setup procedure, I was telling it to setup something like 20GB to use as the system portion. But when the disk went bad and I had to take it to Windows to reformat, it had two partitions, but the first was only 248MB. This repeated every time I tried the install, so something is not right with how it is partitioning. However, I tried the install multiple ways, including just having it use the entire HDD, not setting up a smaller system partition. But after it would crash and I'd go to reformat, it would still have this 248MB partition on it, even when I told it not to partition. Does that seem right? Does it set up a "reserved" partition no matter what?

So at this point I've tried two different HDDs, once with one brand new out of the box, other times with freshly reformatted drives, and it's just not working. Sometimes it would get all the way through the base system install and then get hung during the stage where it was writing the boot portion. Other times it would hang during the system install. At this point I began to wonder if my friends suggesting a bad power supply were right. I even completely removed the other HDDs from the system so they wouldn't be drawing power from the supply. Lather, rinse, repeat... same deal hanging at some point on install. I went downstairs, got out the old PC and removed its power supply. It didn't have all the exact same connectors, but surprisingly (to me, at least) it was able to run the system. All I connected was the motherboard, CD drive, fan, and the HDD. Same deal, hang on install. That doesn't entirely rule out a bad power supply, but in my troubleshooting experience, it makes it much less likely.

Although I had checked the integrity of the image CD I'd created, now I'm wondering if something was wrong with it, or maybe I should just try the full Ubuntu OS instead of just the Server version. So I made an image CD of the normal Ubuntu.

This time, as it booted from the disc, it went through its routine, then I got a message of an "unrecoverable error." I was able to run Ubuntu from the CD, but could not complete the install, on either HDD. Ugh. Anyway, when I do get set up, I am at this point undecided if I should go with the original plan of using Ubuntu Server, or just installing the full OS in case I ever do want to run anything on the PC.

So now I've pretty much ruled out bad HDD, bad power supply, and bad CD image. And, I'm completely frustrated because after reading about Ubuntu and hearing testimonials from good people like Archie, I thought this would be the figurative piece of cake. And, Mrs. Cab is frustrated because I'm pretty much taking up the whole weekend trying to do this "simple computer thing."

Flash back: when I'd had problems with XP in February-ish, I took the PC to Best Buy's Geek Squad because I really had no freakin' idea how to diagnose a non-boot situation. Literally all they said was "motherboard going bad, might be cheaper to buy a new PC." My mobo was under warranty, so I sent it back to Asus. It came back and they said they couldn't find anything wrong with it, and so they reflashed it and sent it back. So I'd thought the Geeks were wrong and the mobo was good goods.

Back to this weekend, now I'm wondering if the Geeks were on to something, and maybe something like a disk/memory controller was failed on the mobo. Since I've tried it every other way I can, and pretty much ruled out every other component (except the processor, let's hope that's not it), I decided to replace the mobo and start over. So I verified the compatibility of my processor and RAM, and went to newegg to find a mobo that wouldn't be too $$$ and would work for me. Especially since I now have this extra 500GB hard drive which means I have one more HDD than I have SATA connectors, so I thought it would be nice to get a mobo with more than 4 SATA connections.

Which all brings us around to:

I ordered a new motherboard last night, should have it in a couple of days and have the server set up shortly thereafter.

Cab
1990 325i(s)
2004 325XiT
Sorry to hear you're having problems sad smiley

Perhaps you can find an answer here; just google 'site:ubuntuforums.org ubuntu 10.4 installation problems' and you may find your problem and solution.
rkj
Quote
Archeo-peteriX
Sorry to hear you're having problems sad smiley

Perhaps you can find an answer here; just google 'site:ubuntuforums.org ubuntu 10.4 installation problems' and you may find your problem and solution.

Sorry for interrupting but can someone tell me (like I'm a two year old) what Cab is trying to do. I think he's trying to have infinite storage for his digital stuff but I don't really know how having his own server will enable him to have that, is there just one kind of server, and what is it really???

This place is kind of its own server, no?

Rick
Not necessarily "unlimited" storage, but here's the basic deal. I have several hard drives that are either full of data, or I am in process of filling them up. As you shoot more and more photos, you need a place to store them. There's only so much room on any one computer or hard drive.

What I have been using is a machine that I built a few years back (you may remember the pain I went through getting it running then, it was with much help from my online buddies that helped me get it going - maybe that should have taught me that I'm really not the best person to build my own computers spinning smiley sticking its tongue out ). My old machine case has physical space for 5 hard drives, but my old motherboard only has the ability to control 4 hard drives. I just bought another hard drive because I thought I needed one. I may or may not end up using it to run my system, but either way, I now have a 250GB, a 400GB, and three 500GB drives. (Two of those 500GB drives are just backing each other up, so they're really the same storage space as one 500GB drive.)

What I'm doing right now (or trying to do) is to set up my old machine in the basement, and all it will do is hold hard drives. That way we can use the new machine to go online and do our "work" without having to worry about running out of storage space on it. All our files will be downstairs, spread out across several hard drives, but will be accessible by the "main" computer over a very small network. Then, when I fill up hard drives on the server, I can just add new ones and archive the old machines (or even add more servers, but I don't see that as really practical).

For my photo work, I am using Adobe's Lightroom program to manage my catalog, and it has the ability to remember where photos are, even if the hard drive they're on has been removed from the system. It will still show you the name of the hard drive, it just has a little red or green light indicating if that drive is connected or not. So if, in a few years, I'm looking for some old photos, Lightroom can tell me where they are, and I can go and reconnect that particular drive.

You could ask, why not just use the space for 4 or 5 hard drives and put monster 2 terabyte or larger drives in them, then in a few years get even larger ones as they become available. After all, storage space is getting cheaper and cheaper. Well, I considered that, but I don't like the idea of having all of my eggs in one digital basket, as it were. I'd rather use smaller (amazing to me that 500GB is considered "smaller") hard drives, in duplicate, so that if a drive crashes, I don't run the risk of losing everything all at once. I probably could accomplish similar or better file security by running some kind of more complex RAID setup, but I haven't gone there yet. Currently I'm just using dual hard drives and when I import photos I automatically back them up to a second drive. That's kind of like a primitive RAID setup, and maybe as time goes on I'll modify how I'm backing up my files.

So, in a nutshell, I'm trying to set up a second machine that will simply house data, so I can access it from the main computer, but have the ability to add or remove hard drives as necessary in the future, looking for long-term storage space, security, and flexibility.

What will probably happen, in reality, is I'll end up with (even more) boxes of old computer parts and various other junk in the basement that we'll never need or look at until our kids have to come deal with all my crap when I buy the farm. Kind of like today, only now it's boxes and boxes of old printed photos and old negatives. grinning smiley

Not sure if that explains it (I tend to ramble on, as you well know). spinning smiley sticking its tongue out

Cab
1990 325i(s)
2004 325XiT
hi

first, please excuse me because i did not read all the above - i am after vacation and pretty tired, drove 7410km smiling smiley
second - i saw something about using a backup disc instead of a mirror system - with linux you can easily configure mirroring.
third, in 2 weeks i made 145gb of photos and movies, so 500gb might not be enough for 3 years smiling smiley

--
A physics truck just turned over outside. There's physics everywhere!
Update time: (Skip to the penultimate paragraph for a quick synopsis, but I warn you, you'll miss all the fun we have in the journey. smiling smileyo )

I received and installed the new motherboard. I was then able to install Ubuntu with very little drama. I'm thinking that the disk controller on the old one is kaput, but that's already filed in the "don't care" folder.

So Ubuntu is/was up and running, I could use the computer, go online, all that happy stuff. But, when I tried to install the Samba package (which I understand is the package that is used to allow Linux machines to interface easily with other OS's), I could not get it set up.

It starts with the simple-sounding instructions on the ubuntu website, which basically say "type the following: 'blah blah blah,' and boom, you're all set and ready to configure Samba." I typed 'blah, blah, blah,' and it went off and opened various files, giving me nice feedback in the terminal window, and it all looked super.

Then the instructions said something like: "go to the configuration file and make the following changes." So I try to do that, and it tells me that I am not the owner of the file, and therefore I cannot make changes to it - it's read-only for me. The properties say that the owner is "root," however when I installed Ubuntu, there is nary a word said about a root account. When I search for "Ubuntu root account" or somesuch, I get many a message that say that Ubuntu is the first (only?) Linux system that does not have or use a root account. It can be enabled, but that is not recommended. Yet all the Ubuntu-specific instructions for setting up Samba don't mention this at all, they imply that it is a piece of cake running Samba on Ubuntu, and never say to enable root. So I have some files that are apparently owned and only changeable by root, and yet I have no root account. The things I read tell me that by using the "sudo" command I will be able to access and change root settings, but that does not seem to be so. I get lost in the Linux-speak because I'm not familiar with the system...

I can see all the hard drives I have installed, and there are all my photo files and other data, but I can't seem to get them visible across the network.

So I got a little annoyed and said to myself, "Self, maybe the Ubuntu Server Edition is a better fit, and the installation of such will guide me through this process." So I uninstalled the full Ubuntu and installed the Server Edition. Beautiful, it installs and I think I'm off to the races. But this time, when it logs in, all I get is command prompts with no instructions for newbies on how to set it up. I flail around for a few minutes, typing random things, but it doesn't seem to understand commands like "you stupid computer, do what I want you to, not what I'm telling you to!" Sigh.

So I re-decide that the full Ubuntu OS is probably better in the long term, because after all, I may want to hook up a monitor to the machine and make some changes once in a while, so I take out the Server Installation CD and play Frisbee (tm) with it.

After installing (again with little to no drama) Ubuntu, I'm seeing that I like the OS, I could see using it instead of Windows, except I like using Photoshop and Lightroom, and those don't work on Linux. But still no joy setting up the server aspect of this machine, which is its raison d'etre, and what's the point of it if it doesn't do what I need it to?

I could probably figure this all out given a few hours of uninterrupted time to work, but I have a 3 y/o and a 2 month old, such time is not to be had in the near future.

So...

I grudgingly admit to myself that it might be best to scrap the Linux idea until I have more time to understand it, and decide that installing WinXP on the server machine would probably be easier in the short term to make the drives accessible to the new Win7 machine. Besides, the new motherboard I installed seemed to fix some of the issues I had, so running XP should theoretically work better now that it doesn't have to deal with a broken whatzit that the old motherboad apparently had. Out comes the XP install disc, and away I go.

But wait, there's more (if you're still with me, have a cookie, you deserve it).

As I'm going through the XP install, it's giving me all kinds of "so and so a file could not be copied, press Enter to retry or Esc to skip... if you skip, the install may or may not be successful." Well, I hit Enter a few times, but no luck, so I start skipping whenever I get this message, which turns out to be quite often, but the names of the files it can't copy seem harmless, as if I know what files are truly necessary and which are bloat... And the coup de grace, eventually I get the famous and comforting Blue Screen of Death before the install can complete. I restart and retry. Same thing.

F#*k!

Now I'm hating Bill Gates even more than usual and re-re-decide that Ubuntu just must be the preferred method for getting this to work, and I can move towards my goal of limiting Gates' control of my life and files. So XP install CD goes back to its own personal purgatory, and the Ubuntu setup CD comes back out. This is now late-ish last night, and I'm running out of patience for the time being. I intend to simply get Ubuntu installed again, rid myself of the scourge of Windows, and start again fresh figuring out how to get the server aspect running at another time.

But this time there is some drama. The install appears to go smoothly, as it had on the several previous occasions I'd installed it, except that it seems to be taking just a hair longer to get everything configured. But it finishes, and gives me the "restart to run Ubuntu" message. When I restart, it just sits there, never actually starting up.

Now I really hate Gates even more, b/c I figure my brief and regrettable moment of XP weakness has screwed something up on either the new motherboard or the hard drive I'm using.

I try installing to a different hard drive, but it hangs up even earlier in the install process, where I select which drive to use for the install. I give it some time and finish up the dishes, but when I come back, it hasn't moved. So, frustrated, annoyed, and irritated, I close up the box, set everything back up on the new PC again so Mrs. Cab can use it this week, and I go to bed.

And that's where we are. Some small victories but overall the war is still ongoing. I had installation success, but couldn't get Samba running. Tried Server edition, it's over my head. Tried XP, it may or may not have broken my system (again). Tried to achieve prior installation success, and have seemingly failed. So the big black box sits at home awaiting my next attempt, which will probably be in a day or two... I think we both need some time to calm down and collect ourselves before we attempt this conversation again. Plus I need to ask some questions on the Ubuntu support forums and do a little reading to see if I can figure out what I'm doing wrong and how a newbie can set up Ubuntu as a server. I really, really hope that attempting the XP install didn't permanently break something. I should be able to simply wipe the drive that I attempted install on, and it should be back to square one, but I haven't tried that yet. I hope that my 250GB drive hasn't been rendered completely dead, but if so, it's not a huge deal, since I have a new 500GB drive that I can use if necessary.

I hope the Pacific is as blue as it is in my dreams. I hope to see my friend again and shake his hand. I hope...

Cab
1990 325i(s)
2004 325XiT
Quote
jaffar
hi

first, please excuse me because i did not read all the above - i am after vacation and pretty tired, drove 7410km smiling smiley
second - i saw something about using a backup disc instead of a mirror system - with linux you can easily configure mirroring.
third, in 2 weeks i made 145gb of photos and movies, so 500gb might not be enough for 3 years smiling smiley

I'm sure with Linux one can do many amazing things... but I don't have the same confidence that I can achieve such success, at least not at my present level of Linux understanding. I may set up better and more efficient backup systems in the future, but I need to win this battle of just getting it set up properly for the first time. Then I may build on that confidence and do more and better things.

I base my 500GB for 3-ish years on my experience of the last couple of years. It may not last that long, and if it doesn't, I'll just add more storage space sooner rather than later. I have found that I really don't care much for video, so other than a little bit here and there of the kids on our mini DV video camera, I/we mostly take still shots. I think video adds up the bytes at a much faster rate than stills. I'm just not interested in making videos, still photos get me much more excited.

500GB is enough for roughly 34k still photos in the RAW format I use, and we shot a little under 10k in our first ~18 months of DSLR use. We now have a second camera that I am using, so I think our burn rate will go up a bit, but I don't think it will double or triple... So anyway, it might only last 2 years, but even that would be fine. I will just archive and get new drives a little sooner. I don't want to use much larger than 500GB drives because I don't want to lose everything all at once if there is a failure. Perhaps, as you say, I will learn to better use Linux and won't need to be so conservative in my backup strategy, and that will be great, but for now this is how I plan to work.

Cab
1990 325i(s)
2004 325XiT
Get a USB external drive. They make excellent backup depots.

alan
Hi Cab!

Just saw this thread here when I read your Buzz. I replied there. I hope you got things working.

I prefer using the desktop version if it's just for home use. It's not even that resource heavy. On idle I only eat around 250MB or so of RAM even with my fully 3D desktop environment:

[i51.photobucket.com] (LARGE PICTURE!!!)

I'm pretty sure if you keep it to the bare minimum, you won't be eating resources. With a desktop version, you can also use it as a backup PC of sorts. On my office, some of my officemates check flash disks that they think may contain viruses on my Ubuntu desktop. It's pretty easy to see hidden Windows autorun and executables this way. smiling smiley You'd be surprised how many anti-viruses fail to detect some of these hidden files!

Check out the links I sent you. Hopefully they will help!
Hey Ernest,

I saw your response, and hopefully it will help, when i get back home and have a chance to try it. I was successfully getting the OS installed, I just had no luck sharing the hard drives on that machine to the Win7 machine... I'll try again. I'm not giving up on this until one of us kills the other. B)

Thanks!

Cab
1990 325i(s)
2004 325XiT
I have one. That's not the answer I'm looking for, though. I would like, ideally, to maintain constant access to multiple HDDs of files without having to swap them in and out of the external enclosure. Ernest gave me some links that hopefully will help and get me to where I think I want to go. I'll beat this thing one of these days. smiling smiley

Cab
1990 325i(s)
2004 325XiT
Well, just make sure you have enough power supply to run all of that. For myself, it seems lately, power supplies have been the bain of my computer exhistance. Either under rated or simply blowing up (4 different machines). But computers are just like cars except you don't get as much grease under your fingernails!!

grinning smiley

alan
rkj
Quote
Cab Treadway
Not necessarily "unlimited" storage, but here's the basic deal. I have several hard drives that are either full of data, or I am in process of filling them up. As you shoot more and more photos, you need a place to store them. There's only so much room on any one computer or hard drive.

What I have been using is a machine that I built a few years back (you may remember the pain I went through getting it running then, it was with much help from my online buddies that helped me get it going - maybe that should have taught me that I'm really not the best person to build my own computers spinning smiley sticking its tongue out ). My old machine case has physical space for 5 hard drives, but my old motherboard only has the ability to control 4 hard drives. I just bought another hard drive because I thought I needed one. I may or may not end up using it to run my system, but either way, I now have a 250GB, a 400GB, and three 500GB drives. (Two of those 500GB drives are just backing each other up, so they're really the same storage space as one 500GB drive.)

What I'm doing right now (or trying to do) is to set up my old machine in the basement, and all it will do is hold hard drives. That way we can use the new machine to go online and do our "work" without having to worry about running out of storage space on it. All our files will be downstairs, spread out across several hard drives, but will be accessible by the "main" computer over a very small network. Then, when I fill up hard drives on the server, I can just add new ones and archive the old machines (or even add more servers, but I don't see that as really practical).

For my photo work, I am using Adobe's Lightroom program to manage my catalog, and it has the ability to remember where photos are, even if the hard drive they're on has been removed from the system. It will still show you the name of the hard drive, it just has a little red or green light indicating if that drive is connected or not. So if, in a few years, I'm looking for some old photos, Lightroom can tell me where they are, and I can go and reconnect that particular drive.

You could ask, why not just use the space for 4 or 5 hard drives and put monster 2 terabyte or larger drives in them, then in a few years get even larger ones as they become available. After all, storage space is getting cheaper and cheaper. Well, I considered that, but I don't like the idea of having all of my eggs in one digital basket, as it were. I'd rather use smaller (amazing to me that 500GB is considered "smaller") hard drives, in duplicate, so that if a drive crashes, I don't run the risk of losing everything all at once. I probably could accomplish similar or better file security by running some kind of more complex RAID setup, but I haven't gone there yet. Currently I'm just using dual hard drives and when I import photos I automatically back them up to a second drive. That's kind of like a primitive RAID setup, and maybe as time goes on I'll modify how I'm backing up my files.

So, in a nutshell, I'm trying to set up a second machine that will simply house data, so I can access it from the main computer, but have the ability to add or remove hard drives as necessary in the future, looking for long-term storage space, security, and flexibility.

What will probably happen, in reality, is I'll end up with (even more) boxes of old computer parts and various other junk in the basement that we'll never need or look at until our kids have to come deal with all my crap when I buy the farm. Kind of like today, only now it's boxes and boxes of old printed photos and old negatives. grinning smiley

Not sure if that explains it (I tend to ramble on, as you well know). spinning smiley sticking its tongue out

I think I've got it; the computer downstairs is the sever and it will be linked up to the main computer upstairs (but how is that link-up done) so you don't have to burden it with all that info, making it slower. That's about it, yes?

I keep on thinking there's a storage place some where in internet heaven that I can store videos and pictures and can stop buying sd cards and flash drives confused smiley after I edit my videos I usually post them somewhere, but youtube and other places like photobucket have limits- size wise and I don't know how safe those places are, but youtube and putfile seem to keep things as far as I can see so far.

All this digital stuff is great fun but eventually you have to have room some where to keep all this stuff. I try to make short films and shoot stills with a moderate amount of pixels, and I'm constantly deleting and culling through my images but the bottom line is.... there's lots of film (and other things) to deal with and that means room somewhere!

I also keep thinking there will be newer storage methods and even the things I use (sd cards and flash drives) are getting cheaper and easier to deal with. I know there's probably a better way for me to go about this but my sd and flash's are easy and cheap for me right now, my movie camara shoots with an sd card, hard drives are a little out of my reach right now but I am looking to the external hard drives in the future, or whatever would serve best. maybe a server like you're trying to get together.

Thanks for the heads up Cab, it's great being on the sidelines watching you guys work at this stuff. I love learning about it; it's kind of my new frontier thumbs up smileys with beer thumbs up

Rick
rkj
Cab, after reading all the posts you're doing the right thing, take a break and sit back. Sometimes a few steps back are the only way you're going to get any forward progress. I've had a few projects like that, stay with it and keep us posted- it's getting interesting thumbs up

How do your two computers talk to each other, wireless, hardwired, magic?

Rick
Quote
rkj
How do your two computers talk to each other, wireless, hardwired, magic?

I'm working on the magic, but it's not been successful as of yet. So until I get that down, they're hardwired to each other by virtue of both being plugged into the same router. Now I need to make sure I understand how to get them to "see" each other.

Ernest's tips he sent me yesterday, combined with something another friend passed along to me, make this seem so ridiculously easy, I think I just thought it was harder than it should be. I didn't have a chance to try again last night, but whenever it is I get around to my next session, it really seems like it should be a piece of cake. It's just something so easy that I didn't try it before, I was busy trying to make it harder on myself than it really is. We shall see...

Cab
1990 325i(s)
2004 325XiT
By the way, if you are using Ubuntu Lucid Lynx, I had to do this for the Windows computers to see my shares:

  1. Open a terminal and type "sudo gedit /etc/samba/smb.conf" If you don't have a GUI installed (server edition) type "sudo nano /etc/samba/smb.conf" then press enter.
  2. You will see a configuration file. Find the line that says:

    # What naming service and in what order should we use to resolve host names
    # to IP addresses
    ; name resolve order = lmhosts wins bcast host


    Make sure the name resolve order is the same as the one I posted above. The order of the names is important here. You can just copy and paste the above.
  3. Save and close the file.
  4. Install Winbind by typing "sudo apt-get install winbind" in the terminal.
  5. Restart your computer.

Windows 7 computers seem to have some quirks too. From another forum:

Quote

Windows 7 configuration:

First of all, remove Windows Live ID Sign-in assistant if you have it installed. More information here: [social.technet.microsoft.com]

Windows 7 seems to be bent on wanting you to use "HomeGroup" file sharing, but this will not work with Samba, it will only work with other Windows 7 computers, and you're probably better off disabling it if you need things to work correctly in a mixed network.

In order to get simple file sharing to work, you'll need to click on the "Change advanced sharing settings", and make sure you have network discovery, file and printer sharing, and public folder sharing enabled in the "home or work" profile (also, make sure that "home or work" is labeled as your current profile). You should be able to use 128 bit encryption, and use either enable or disable password protected shares (depending on how much security you desire).

Then, go to "Change adapter settings", right click on your connection and select "properties". Click on the "Internet protocol version 4 (tcp/ipv4)" and select "properties". Then click on "advanced ..." (at the bottom), and select the "WINS" tab. Make sure that "Enable LMHOSTS lookup" is enabled (no need to import), and make sure that "default" is selected for NetBIOS setting.

Now, go to a folder (try the public folder first), and right click on it. Select "properties", and click on the "sharing" tab. Click on "Advanced sharing", and put a check mark in "Share this folder". Change the "share name" if you like, and click on the "permissions" button. Make sure you have permissions set as you desire (a checkmark in "change" = writeable). If you've enabled password protected shares, you'll have to tweak settings here to make sure that your Ubuntu user is included for permission.

In some situations, you will find it necessary to add your Ubuntu username and password as an account on your Win7 machine. This may seem tedious, but that's the way secure Samba servers work.

Once this is all set, then your folder will show "shared" and it's network path will be displayed.

I hope this helps.
Quote
rkj
I think I've got it; the computer downstairs is the sever and it will be linked up to the main computer upstairs (but how is that link-up done) so you don't have to burden it with all that info, making it slower. That's about it, yes?

It's not just slowing it down with all that data, it's simply the amount of storage space. My new "main" computer has a 500GB hard drive. That's not nothing, but it will fill up relatively soon with all the photos we shoot. You can argue that I could/should be deleting a lot more photos than I do, but I tend not to, I just save everything unless it's really horrible. But anyway, having a server with several slots for hard drives means I can fill them up, then remove, replace, and archive them, and I turn 34k photos into a small box on a shelf in the basement. Then my catalog software - Lightroom - can tell me which hard drive to remount when and if I want to access a particular photo sometime in the future. This lets me keep the main computer physically small and lean, and have almost unlimited storage flexibility as well.

This is also mainly for storage of my RAW photo files, kind of like the negatives from film days. Once I edit a photo and "finish" it, it becomes a JPG, and those files are much smaller. That format is what gets uploaded to the photo sharing sites (photobucket, facebook, etc). The RAW files are much, much larger, and so they need more room to keep them. I could trash them once I make a JPG, but that limits the possibility of ever going back and making any changes, should I desire. Always keep your negatives. spinning smiley sticking its tongue out

Quote

I keep on thinking there's a storage place some where in internet heaven that I can store videos and pictures and can stop buying sd cards and flash drives confused smiley after I edit my videos I usually post them somewhere, but youtube and other places like photobucket have limits- size wise and I don't know how safe those places are, but youtube and putfile seem to keep things as far as I can see so far.

All this digital stuff is great fun but eventually you have to have room some where to keep all this stuff. I try to make short films and shoot stills with a moderate amount of pixels, and I'm constantly deleting and culling through my images but the bottom line is.... there's lots of film (and other things) to deal with and that means room somewhere!

I also keep thinking there will be newer storage methods and even the things I use (sd cards and flash drives) are getting cheaper and easier to deal with. I know there's probably a better way for me to go about this but my sd and flash's are easy and cheap for me right now, my movie camara shoots with an sd card, hard drives are a little out of my reach right now but I am looking to the external hard drives in the future, or whatever would serve best. maybe a server like you're trying to get together.

Yes, SD cards and flash drives are nice, but after a while you start to have dozens of them, and even though they're small, if you're using them as permanent storage, it gets very difficult to keep track of them all. I think it's better to have a more streamlined approach.

There certainly are places in the internet ether where you can store files. In fact, I've read some photographers (mostly pros who need file security to make their living) will back up files in as many as 3 places locally and then also remotely through one of several online storage services. As you mention, if you stick with free sites like youtube or photobucket, there is a limit on both the size of each file, and the total amount of data you can store, plus bandwidth limits so you can't always download all of it at once (if you need to).

Obviously I put photos online to share them with people, but all my backup is done locally. Maybe someday I'll expand that to some online storage as well. Of course with anything online, you have to keep security in mind, can they be accessed by just anyone, etc?

Quote

Thanks for the heads up Cab, it's great being on the sidelines watching you guys work at this stuff. I love learning about it; it's kind of my new frontier thumbs up smileys with beer thumbs up

Rick

No problem. Despite my frustrations, I do enjoy doing projects like this. They help me learn more, and when I'm done I have a greater appreciation for how it all works. Kind of like replacing subframe bushings, eh? tongue sticking out smiley

Cab
1990 325i(s)
2004 325XiT
Quote
Ernest
By the way, if you are using Ubuntu Lucid Lynx, I had to do this for the Windows computers to see my shares:

Open a terminal and type "sudo gedit /etc/samba/smb.conf" If you don't have a GUI installed (server edition) type "sudo nano /etc/samba/smb.conf" then press enter.
You will see a configuration file. Find the line that says:

# What naming service and in what order should we use to resolve host names
# to IP addresses
; name resolve order = lmhosts wins bcast host

Okay, this is where I seemed to run into problems before. I followed all the instructions I read on the ubuntu and samba websites, using this sudo command, but I was not allowed to edit the smb.conf file. It said I wasn't the owner and therefore couldn't save changes. But I never used the exact command "sudo gedit /etc/samba/smb.conf"

Maybe that command gives me permission to edit that file directly? I'll certainly try it.

Quote

Windows 7 computers seem to have some quirks too. From another forum:

snip...

I hope this helps.

It seems to be exactly what I need. Thanks so much for the help, I will try this as soon as I can, possibly tonight or later this week, depending on my energy level. You rock!

Cab
1990 325i(s)
2004 325XiT
Yes. "Sudo" basically gives you admin privileges for a short period. It will prompt you for your password when you enter the command.

I have to add:

"gedit" opens the graphical text editor. It's similar to "notepad" in Windows.

"nano" opens the command line editor. It's similar to editing txt files inside the Windows Command Prompt.

Also note that in Linux, directories are case-sensitive.

I hope this helps. smiling smiley Learning Linux isn't as hard as some people think. smiling smiley



Edited 1 time(s). Last edit at 08/10/2010 08:38AM by Ernest.
My understanding after reading the tutorials on the ubuntu site had been that once I used the "sudo" command to get and install samba, then I could just open and edit the smb.conf file. It did not say that I had to use "sudo" to open the config file. I'm sure that this is at least part of my problem. But... as you indicated, and another friend of mine sent me a link to, it should be even easier than that. Since I can see the folders I want to share, I shouldn't have to manually mess with samba at all. Just right click and go into the sharing properties. If samba is not already configured, it should download and configure it for me. So I am looking forward to getting a chance to play with it more, and I'm sure I'll need your tips on getting Win7 to play nice with Linux as well.

I really am optimistic that I'll be able to get this working soon. Thanks again, and I can't wait to be able to report on my enormous success. hot smiley

Cab
1990 325i(s)
2004 325XiT
Yes it really is that simple. However, I found on some networks, I still have to tweak the samba config file a bit. Just changing the order of the name resolve order to "; name resolve order = lmhosts wins bcast host " does wonders. You might need Winbind too. These are all backup plans in case the simple right click and share won't work on the first time. Once everything is setup, you can share as many folders as you want by simply right clicking and setting them to be shared. smiling smiley

Oh by the way, what file system format is your shared drive?
Quote
Ernest
Oh by the way, what file system format is your shared drive?

The drives I plan to share are NTFS.

So, my problem as of last night appears to be hardware-related, or at least that's what it seems. I formatted and started over installing Ubuntu on my 250GB drive. It went through the whole process, got to the end and said "restart computer to boot in Ubuntu." I restarted, and after the BIOS start-up screen, I got nothing. Just a black screen with a blinking cursor about 3-4 lines down from the top. This is where I got stuck over the weekend after my failed attempt to install WinXP.

I thought maybe that 250GB drive was bad, even though when formatted, it said that it was healthy. So, I removed that drive and did the Ubuntu install on a different one, the new 500GB drive I bought last week. Same thing happened. The install said it completed, got to the "restart" message, but when restarted, I just get this blank screen with a blinking cursor. No command prompt or anything, just a little blinker. I've checked my BIOS settings, and I'm using the correct boot order, CD drive first, followed by HDD, and I made sure that the correct HDD was selected. What I didn't try yet is to go back and make sure that the BIOS has all the initial default settings configured and/or use the CD that came with the motherboard to reset it to factory condition.

Would/could trying to install XP have changed any BIOS settings that are now causing me to have a no-boot condition? I was able to install and run Ubuntu previously, but ever since trying XP, I've not been able to boot into Ubuntu (except from the CD).

Cab
1990 325i(s)
2004 325XiT
Usually installing XP after installing Ubuntu tends to break things. Windows doesn't like to share the same computer. smiling smileyo Generally, you'd want to install XP first then Ubuntu.

Your boot loader might have been corrupted. Try to restore grub:

[ubuntuforums.org]
Sorry, only registered users may post in this forum.

Click here to login