[Ksummit-2012-discuss] PCI breakout session

James Bottomley James.Bottomley at HansenPartnership.com
Mon Jun 18 21:23:25 UTC 2012


On Mon, 2012-06-18 at 16:15 +0300, Michael S. Tsirkin wrote:
> On Mon, Jun 18, 2012 at 01:53:52PM +0100, James Bottomley wrote:
> > On Mon, 2012-06-18 at 14:43 +0300, Michael S. Tsirkin wrote:
> > > On Mon, Jun 18, 2012 at 12:25:45PM +0100, James Bottomley wrote:
> > > > On Mon, 2012-06-18 at 11:08 +0300, Michael S. Tsirkin wrote:
> > > > > An interesting related problem is hot-plugging complex bridged hierarchies - esoteric
> > > > > on real hardware but easy and sometimes useful in a VM.
> > > > > It would seem that at least for that space, adding support
> > > > > for moving resources around could be useful. For things like IO it
> > > > > only seems possible to do with cooperation from the drivers - and at least for
> > > > > VMs that's not such a big deal as we only emulate a handful of devices.
> > > > > Or do we need to try and solve it generally straight away?
> > > > 
> > > > What's missing?  Some of the parisc systems end up with crap (or no)
> > > > window assignments when left to firmware, so we do a lot of re-layout of
> > > > PCI device resources (see for example drivers/parisc/dino.c).
> > > > 
> > > > This shouldn't be generic: you need a lot of platform internals
> > > > knowledge to muck around with PCI bus resource assignments, so I'd need
> > > > convincing there should be a more generic interface than the existing
> > > > pci bios fixup callbacks.  Is the problem just that you can't use them
> > > > because we don't have a PCI driver for virtual busses?
> > > > 
> > > > James
> > > 
> > > Also, assuming we write our own driver with its own
> > > allocator and balancer, we'd like some clean interface
> > > to tell a virtio pci device that we are moving its BARs
> > > while it's active.
> > 
> > I think this may be the core of my misunderstanding about what you're
> > trying to do: why is it active?  If you've just hotplugged the bridge,
> > the pci bios routines let you set up (or relayout) the resources of
> > attached devices *before* any drivers attach.  You fix up all the issues
> > in the windows (and even sometimes widen the window) then you let the
> > drivers at the devices.
> > 
> > James
> 
> Yes but as you add and remove devices the space becomes
> fragmented, while a bridge can only claim a contigious
> region for devices behind it. If we could move devices
> without unbinding drivers, it would help pack them more tightly.

Hm, OK, so your bridge only has a single window?  (Ours can have up to 4
although we rarely use them all).

> It's a niche problem since most people don't have that
> many devices.
> 
> I hope that explains what I was trying to say.

Sort of ... we have this type of problem on 32 bit systems where
physical space is at a premium, but on 64 bit ones, we just leave a huge
amount of space for bridge windows, so in practice it doesn't arise.

James




More information about the Ksummit-2012-discuss mailing list