Yeah, I'm definitely interested in very quickly trying out a few things to see what difficulties we come across. I definitely don't want to throw out anything without trying it.
A few thoughts.
1. Point and shoot cameras continue to get better and better. We have 16mp compacts out there (with the usual quality caveats). In a year or two, single-camera solutions will be the norm.
2. Lasers are really cheap. I ordered a set of line-generating lasers for $3 each from DealExtreme. I'll order more if they turn out to be any good.
3. The green channel of a Bayer-patterned sensor has the highest resolution.
4. It's easier to deal with sequential images from one camera than many images from many cameras, potentially with sequential errors.
5. The basic image geometry for the laser beams is just trigonometry, once you find the line.
6. ST already dewarps using two lines (maybe we can re-use some of ST)
However, I do like the directness of your idea - want to sense the thickness of a book? Take a picture of it!
Honestly, I think the challenge will be the same in both instances, but my hand-waving intuition is that sensing a bright green line may be easier than sensing the edge of a book against some background.
Do you have any time in the coming weekend? I'd bet we can get a lot of sample images done if we were working together at my place.
Methods To Sense The 3D Surface/Structure Of A Book
Moderator: peterZ
-
daniel_reetz
- Posts: 2812
- Joined: 03 Jun 2009, 13:56
- E-book readers owned: Used to have a PRS-500
- Number of books owned: 600
- Country: United States
- Contact:
-
spamsickle
- Posts: 596
- Joined: 06 Jun 2009, 23:57
Re: Methods To Sense The 3D Surface/Structure Of A Book
I should be available all day Saturday, and Sunday until about 2:30, unless you end up having to work anyway, in which case I was busy too, maybe some other time.
Re: Methods To Sense The 3D Surface/Structure Of A Book
Yea scattering will be a problem, you'll need good line generators. Also green lasers I think are too bright, so they'll create pretty "fuzzy" lines. You want fairly low power dim clean red line generators. IMO
What advantage this would have is that it's essentially doing what google is doing, except removing the IR. This could be done with IR, just take a cheap digital camera and take it apart, remove the IR blocking filter infront of the sensor and blamo, you have a digital camera capable of seeing IR lines, then you'd need a IR laser grid generator... still thats probably more expensive.
$8 line generators, at most 10 of them for a grid, or maybe there exists cheaper solutions to generate a checkerboard type grid with lasers, then photograph it in visible light.
The key here rests with software, is there any open source algorithms to dewarp with this projected grid pattern?
What advantage this would have is that it's essentially doing what google is doing, except removing the IR. This could be done with IR, just take a cheap digital camera and take it apart, remove the IR blocking filter infront of the sensor and blamo, you have a digital camera capable of seeing IR lines, then you'd need a IR laser grid generator... still thats probably more expensive.
$8 line generators, at most 10 of them for a grid, or maybe there exists cheaper solutions to generate a checkerboard type grid with lasers, then photograph it in visible light.
The key here rests with software, is there any open source algorithms to dewarp with this projected grid pattern?
-
daniel_reetz
- Posts: 2812
- Joined: 03 Jun 2009, 13:56
- E-book readers owned: Used to have a PRS-500
- Number of books owned: 600
- Country: United States
- Contact:
Re: Methods To Sense The 3D Surface/Structure Of A Book
Why do we need a grid? Right now I am thinking two lines are enough.
Re: IR cameras, yep, I'm familiar.
But I don't want to be using IR because we'll likely be infringing on Google's patent, and that's not going to help us stay in business.
I'm not yet worried about the line quality at all. We can improve it somewhat with a baffle and/or pinhole. But I'll order a couple of red line generators anyway. Even if it is fuzzy, we can threshold it or use that information to produce a better line (it will get wider as the angle gets more oblique).
Re: IR cameras, yep, I'm familiar.
But I don't want to be using IR because we'll likely be infringing on Google's patent, and that's not going to help us stay in business. I'm not yet worried about the line quality at all. We can improve it somewhat with a baffle and/or pinhole. But I'll order a couple of red line generators anyway. Even if it is fuzzy, we can threshold it or use that information to produce a better line (it will get wider as the angle gets more oblique).
-
Anonymous1
Re: Methods To Sense The 3D Surface/Structure Of A Book
How is this done so quickly? http://www.youtube.com/watch?v=tCOXC5PTJj8
I'm not sure about the hardware, but just from watching, you can tell that it's really high-speed line tracing. I think that the whole process can be simplified and done completely without lasers and overkill hardware. How would a human dewarp mentally? I would determine the height of a line of text, it's relative offset from the imaginary line that determines the edge of a page, and the spacing between neighboring lines. Sure it's hard to code, but it's not impossible. Tulon's coupled snakes approach does most of this quite well (it just slips up when choosing representative lines
)
From all this data, you could tell what should be done to correct a page, so the reverse could generate a 3D model. Just a thought...
The easiest approach for me would be to just setup a few cameras looking at the book perpendicular to the text, so that you could have two images of the curvature of the whole book's content. Then, you just need the dimensions, the lens distortion for all the cameras, precise distances for all the components, and some good extrapolation methods. From there, it's all good.
I'll play with this method (if any at all), as I don't have the money to buy sensors and IR cameras...
I'm not sure about the hardware, but just from watching, you can tell that it's really high-speed line tracing. I think that the whole process can be simplified and done completely without lasers and overkill hardware. How would a human dewarp mentally? I would determine the height of a line of text, it's relative offset from the imaginary line that determines the edge of a page, and the spacing between neighboring lines. Sure it's hard to code, but it's not impossible. Tulon's coupled snakes approach does most of this quite well (it just slips up when choosing representative lines
)From all this data, you could tell what should be done to correct a page, so the reverse could generate a 3D model. Just a thought...
The easiest approach for me would be to just setup a few cameras looking at the book perpendicular to the text, so that you could have two images of the curvature of the whole book's content. Then, you just need the dimensions, the lens distortion for all the cameras, precise distances for all the components, and some good extrapolation methods. From there, it's all good.
I'll play with this method (if any at all), as I don't have the money to buy sensors and IR cameras...
Re: Methods To Sense The 3D Surface/Structure Of A Book
Re: the video, they're using a ultra-high speed HD camera + the IR 3d sensing method to in real time map the 3d structure of every fame of the high-speed camera, then using software to figure out which frame is best for each page after dewarp. I think.
This seems fairly impractical since you can't really flip accurately and not miss pages, and to write software to know where a page miss was could be difficult. It's just a "proof-of-concept", probably eventually be coupled with a robotic page turning apparatus that can turn at very high speeds... I've seen it a while ago but didn't seem very practical.
This seems fairly impractical since you can't really flip accurately and not miss pages, and to write software to know where a page miss was could be difficult. It's just a "proof-of-concept", probably eventually be coupled with a robotic page turning apparatus that can turn at very high speeds... I've seen it a while ago but didn't seem very practical.
Re: Methods To Sense The 3D Surface/Structure Of A Book
Google's patents probably mostly pertain to the software/math side, do you know what the patent #'s are? Even still staying away from IR will reduce costs, imo. So thats another benefit. IR just allows you to go faster since you can shoot one picture.daniel_reetz wrote:Why do we need a grid? Right now I am thinking two lines are enough.
Re: IR cameras, yep, I'm familiar.
I'm not yet worried about the line quality at all. We can improve it somewhat with a baffle and/or pinhole. But I'll order a couple of red line generators anyway. Even if it is fuzzy, we can threshold it or use that information to produce a better line (it will get wider as the angle gets more oblique).
I wonder if you can get a glass filter that blocks everything but the nm range of the laser light, and a filter that blocks the laser light only... then you could basically do what google does without IR, project the laser light continuously and take both pictures at once. That would dramatically increase costs though for those filters.. heh.
I'm personally for doing it with minimal amount of pre-processing of the images as possible... but writing the software to accept everything from poor quality lines that have lots of bleed light and not positioned 100% accurately to perfect grids would be good.
as for the grids... I donno much about the algorithms involved so I can't say.. I just know that sensing depth usualy involves grids, like the Kinect sensor and google's dewarping tech both use grids... the work being done on the software for the scannable checkerboard to dewarp the page...
My thoughts is if you project a checkerboard pattern, a grid, the same algoritms that is used in that software might work?
It really boils down to the software, do we have an idea how to create it?
-
Anonymous1
Re: Methods To Sense The 3D Surface/Structure Of A Book
You can probably just dump the final page output and flip through the book a few times, and then just use OpenCV to quickly look for duplicates. But I do agree, this method is useless for older books.kasslloyd wrote:This seems fairly impractical since you can't really flip accurately and not miss pages, and to write software to know where a page miss was could be difficult. It's just a "proof-of-concept", probably eventually be coupled with a robotic page turning apparatus that can turn at very high speeds... I've seen it a while ago but didn't seem very practical.
Re: Methods To Sense The 3D Surface/Structure Of A Book
The cost of the high speed camera though could buy a bunch of conventional high quality book scanners though.. heh.
So unless you was going for (a) limitless budet for digitizing and (b) completely automated system with minimal staff.. I also couldn't see justification in purchasing such a camera...
So unless you was going for (a) limitless budet for digitizing and (b) completely automated system with minimal staff.. I also couldn't see justification in purchasing such a camera...
Re: Methods To Sense The 3D Surface/Structure Of A Book
the software seams easy,
but i am also thinking that a line is not enough, if the camera is 100% perpendicular to the book and the line, then the camera sees just an straight line, so to see the wave we need to look from the side, but in which angel? and this angel is creating an new distortion (as long you book scaner and cameras are on fixed places it can be corrected witch one measurement)
we need dots like this one http://cgi.ebay.de/6-1-Laser-Pointer-Ze ... 4394wt_907
even a random pattern could be help, then we need just train the software with a none warped page what the pattern looks like
pattern matching can work with very low signal strange, so it will be maybe possible to dim the laser and later remove from the image, this would allow to take one image per page
software:
http://biocomp.cnb.uam.es/~iarganda/bUnwarpJ/ if its posible to gain the laser signal (by software) and reduce the page content, if you have a random pattern and a image which is none wrapped and a page which is wrapped bunwrapj will find the deformation map
but it can say, this will need cpu time
but i am also thinking that a line is not enough, if the camera is 100% perpendicular to the book and the line, then the camera sees just an straight line, so to see the wave we need to look from the side, but in which angel? and this angel is creating an new distortion (as long you book scaner and cameras are on fixed places it can be corrected witch one measurement)
we need dots like this one http://cgi.ebay.de/6-1-Laser-Pointer-Ze ... 4394wt_907
even a random pattern could be help, then we need just train the software with a none warped page what the pattern looks like
pattern matching can work with very low signal strange, so it will be maybe possible to dim the laser and later remove from the image, this would allow to take one image per page
software:
http://biocomp.cnb.uam.es/~iarganda/bUnwarpJ/ if its posible to gain the laser signal (by software) and reduce the page content, if you have a random pattern and a image which is none wrapped and a page which is wrapped bunwrapj will find the deformation map
but it can say, this will need cpu time