[Ksummit-2012-discuss] PCI breakout session

[Michael S. Tsirkin]

On Mon, Jun 18, 2012 at 12:25:45PM +0100, James Bottomley wrote:
> On Mon, 2012-06-18 at 11:08 +0300, Michael S. Tsirkin wrote:
> > On Mon, Jun 18, 2012 at 08:41:43AM +0100, James Bottomley wrote:
> > > On Mon, 2012-06-18 at 07:42 +1000, Benjamin Herrenschmidt wrote:
> > > > (snip general approval)
> > > > 
> > > > So folks, what should be the format ? Do we organize a mini-summit
> > > > proper with an agenda or we just breakout during KS itself ?
> > > > 
> > > > Ted, does the venue has facilities for us to do something maybe the day
> > > > before ? Or we just do it informally in the corridors ? :-)
> > > > 
> > > > I think the main thing to discuss is rework & cleanup of hotplug which
> > > > is a large impact (PCI boot code as well) if we want to do it properly,
> > > > which means having essentially the same code path overall to deal with
> > > > normal PCI probe and hotplug.
> > > > 
> > > > The main issue I see in term of infrastructure is how we rely on
> > > > initcall "levels" and ordering of things at boot to do things like
> > > > probe, then assign resources, then fixups, then bind drivers. It's
> > > > fragile and doesn't work properly with hotplug, meaning that hotplug
> > > > uses a subtely different ordering (and totally fails to call the final
> > > > fixups).
> > > > 
> > > > The "right" approach is to probably move the resource assignment to
> > > > between the initial scan pass and the "adding" of devices (in our PCI
> > > > stack terminology, adding -> register with the driver model), and
> > > > naturally have the final fixups called right before the latter. But this
> > > > will have some sort of impact on all archs so we probably want to do
> > > > quite a bit of code auditing first.
> > 
> > An interesting related problem is hot-plugging complex bridged hierarchies - esoteric
> > on real hardware but easy and sometimes useful in a VM.
> > It would seem that at least for that space, adding support
> > for moving resources around could be useful. For things like IO it
> > only seems possible to do with cooperation from the drivers - and at least for
> > VMs that's not such a big deal as we only emulate a handful of devices.
> > Or do we need to try and solve it generally straight away?
> 
> What's missing?  Some of the parisc systems end up with crap (or no)
> window assignments when left to firmware, so we do a lot of re-layout of
> PCI device resources (see for example drivers/parisc/dino.c).
> 
> This shouldn't be generic: you need a lot of platform internals
> knowledge to muck around with PCI bus resource assignments, so I'd need
> convincing there should be a more generic interface than the existing
> pci bios fixup callbacks.  Is the problem just that you can't use them
> because we don't have a PCI driver for virtual busses?
> 
> James

Also, assuming we write our own driver with its own
allocator and balancer, we'd like some clean interface
to tell a virtio pci device that we are moving its BARs
while it's active.


-- 
MST