I've done something like this by using curl. First, if you give an ftp:// directory URL to curl -ls, it will nicely print the list of filenames in the directory. Since I believe all package names are followed by "-$version" for their filename, you can use a loop with grep/egrep to get the URLs for each package.

for rpm in `curl -ls $REPO | egrep '^(perl|mysql)-[0-9]'`
    do wget $REPO$rpm
done

Granted I've only used this for a few files to work around the removal of --nodeps from livecd-tools; I'm not sure how efficient this will be for a whole list of spin packages. Nevertheless, I hope this helps.

(Reply) (Thread)

For each package, tack on a "-[^-]+-[^-]+.rpm", since other than the package name you can only have those two hypens in the RPM name. If you're on x86_64, ending it with .x86_64.rpm would be needed since i386 is there as well. Hope this helps.

--MathStuf

(Reply) (Thread)

I would use the DVD or cdrom you installed from. Make this act like a local mirror and spin your image from that. While you won't get updates... (unless you mirror that locally).. it would give you a spin for that.

A ludicrous way of doing this is having a dedicated local mirror box. You power it on every now and then and it sync's up with whats on the mirror site. You then have it have a cobbler server on it and just run your special kickstart to get your system.. this way you are within 1 week of latest if you need to do a rebuild and you could take the box to do an installfest.

(Reply) (Thread)

Can you just run them back through yum with the download-only add-on?
Then you get to "rise above" the transport issue and could probably utilize yum's intelligence (for resuming downloads & following through on dependencies too).

jay

thecapacity -> gmail

(Reply) (Thread)

Feed /root/anaconda-ks.cfg to pungi and it should handle the rest.

(Reply) (Thread)

Look at how preupgrade does it. It uses yum to download the content you would want, and creates a _local_ yum repo out of that content.

(Reply) (Thread)

susmit had hinted that he was working on something similar,might ping him later.

~sankarshan

(Reply) (Thread)

Greetings,

I'm a member of the OpenVZ Project community and I've been involved with building OS Templates of various Linux distributions. Why do I mention this and how is it relevant? Because building an OpenVZ OS Template is similar to building a customer spin of a distro. Stick with me for a few paragraphs.

A fellow by the name of Robert Nelson took a package from Debian named apt-cacher and rewrote it to be more general purpose for use with OpenVZ vzpkgcache. vzpkgcache is a utility that uses an OS Template Metadata package to build an OS Template. What's a metadata package? Basically it is a small package that contains information about three things:

1) Where on the internet are the package repositories found for whatever distro you are working with - for example the metadata package for Fedora includes information about repos for Fedora 7, 8, 9 and 10... for i386 and x86_64

2) What packages you to make up your OS Template - for example there are definitions for "small", "minimal" and "default". These definitions are just text files with a list of desired packages

3) A list of OpenVZ dummy packages that replace distro specific packages for the small handful of packages that aren't compatible with OpenVZ. For example, udev didn't work well in a container so there is often a dummy package to replace it named vzdev

pkg-cacher is used as a general purpose package caching system for use with vzpkgcache. It runs on a port of your choosing and after setup it can grab packages needed for building CentOS, Debian, Fedora and Ubuntu.

Not only can pkg-cacher be used for building OS Templates with vzpkgcache... it can also be used by package managers (yum and apt-get) on client machines and/or OpenVZ containers. The package caching system will appear to be a local repo copy when in reality if it doesn't already have the package you are looking for it'll grab it on demand. OS Templates made with Robert's modified vzpkgcache are configured to use the package caching system... so when you create a container from the OS Template, it'll use the cache. This works with CentOS, Debian, Fedora, and Ubuntu clients. He moves existing repo definitions (.repos and source.lists) to .disabled and supplies a new repo definition that refers to the caching systems simulated repos.

I'm thinking that pkg-cacher could be used for what you want and if not, it could be modified to do what you want. I think pkg-cacher is a package that a lot of Fedora and CentOS users would be interested in. Anyone who rsyncs one or more distro mirrors for local use would love to be able to replace a full copy of the mirror with a smart caching system like pkg-cacher. Considering the huge size of some distros these days (all of Fedora 10 is between 11,500 [i386] and 14,500 [x86_64] packages), I'm surprised everyone isn't using pkg-cacher.

You can find its homepage here:
http://gforge.opensource-sw.net/projects/pkg_cacher/

You can Fedora 10 packages for it here:
http://repo.opensource-sw.net/fedora/10/i386/

pkg-cacher has made it really easy to build over 54 different OS Templates for OpenVZ. For a screencast on it, see:
http://montanalinux.org/vzpkg2-screencast.html

If you have any questions, feel free to email me.

TYL,
--
Scott Dowdle
704 Church Street
Belgrade, MT 59714
(406)388-0827 [home]
(406)994-3931 [work]
dowdle@montanalinux.org

(Reply) (Thread)

set up the repos you want to use in a dir and make a custom yum.conf pointing to this dir

cd somedir
yumdownloader -c custom.conf acl
etc.

then your don't have to mess with package versions, yumdownloader will do all the action for you.

(Reply) (Thread)

Being in the middle right now of recovering all of the packages that I didn't get in the reinstall, what has stuck me is this question: what about changes to default applications? How do you know what is different in defaults between releases without doing a three-way diff between your installed packages, Fedora N-1 comps.xml, and Fedora N comps.xml?

(Reply) (Thread)

I would definitely use one of the tools provided by the distribution rather than a script written and maintained elsewhere. You probably don't want to build up processes that rely on scripts that are more likely to go defunct.

Pungi and Revisor are both excellent options. I prefer Revisor simply because it allows more flexible selection of mirrors from which to download (AFAIK. maybe pungi is more capable that I think). This allows you to use a local repository as one source, and avoid downloading the bulk of packages when you refresh your local respin.

Both Pungi and Revisor will rebuild the initrd and install images, which is something of a time-consuming process. Unless you need them rebuild (usually in order to include newer drivers or newer kernel in general), you can opt to skip the "buildinstall" stage. In pungi, I believe that you can use "-G -C" to only gather packages and "createrepo". For revisor, you'd install the additional revisor-reuseinstaller package, and use the "reuse" option.

Let me know if you'd like more detailed info... Maybe I could write up a brief howto in my own journal.

(Reply) (Thread)

Based on other suggestions from a few days ago, I'm using yumdownloader, which works great so far.

(Reply) (Parent) (Thread)