Consultancy

I have a permanent job at the NetCracker‘s System Performance group. Recently I was offered to do one day job outside, on-site in another company, which coincidentally has an office close to NetCracker’s Moscow office. It was an opportunity to apply my skills in a completely different situation which I couldn’t miss; plus I’ve never done public presentations before and this was a good occasion to practice that. Here I’d like to write down some notes how the event went.

2 or 3 weeks before the event I’ve got the list of topics to work on. It included several Java and Oracle subjects. I’ve offered couple of topics myself too. The preparation went well although it took a lot of time to make some slides (which I don’t really like doing hence as a result slides look clumsy. Just as my blog :)) and to put together meaningful examples. In fact I didn’t prepare everything in the volume I thought about initially because of the time – and as it turned out it was right decision. I didn’t practice presentations even though I tried. Each time I’ve tried I took different ways to describe same things so I’ve decided to go unprepared and see how it will go. Since I have clear understanding of what I’m talking about and had a plan in slides it wasn’t so hard. What was a bit of a problem I didn’t know what to expect from the audience and how to keep the balance between interesting and boring slides. Both problems didn’t materialize and were somehow avoided :) Must be because of simplicity I think. I tried hard to make it simple and keep what’s important in my opinion.

So I took a day off and went to present. It started badly. My notebook is a tiny Aspire S3 with only HDMI output available. When I tried to connect it to the projector, it just didn’t work. I spent half an hour trying different things until when I did something wrong and the screen went black with nothing on the projector screen. After a reboot laptop didn’t boot up properly, and I had no other choice than to use safe mode and apply System Restore which added another 10 minutes. Luckily restore finished well. It must have been Friday 13th impact :) but no. Turned out the plug I tried was not for the projector, but for the speakers!! The right plug was hided behind the screen wall. Fortunately there was another laptop available with DVI port, which I used to show slides. Mine notebook was behind the wall connected via HDMI, so each time I needed to show a demo I had to go up, switch the projector’s input and hide myself from the audience (the only thing visible was my back) appearing from time to time. It was funny.

First part was about JVM performance. I talked about general HotSpot JVM features such as just in time compiler and garbage collection. The latter was the main topic of this part. I covered JVM memory layout, GC types, their high-level algorithms and options to tune them based on my experience. In the end of the section there was a demo for out-of-the-box monitoring/debugging/analyzing and third-party tools such as Memory Analyzer. With only 16 slides it took almost 2 hours to talk and answer questions. I didn’t anticipate it that long, and expected to fit into an hour or so. Apparently I can talk for a long time :)

Next topic was Oracle JDBC. I’ve started with an overview of what is end-to-end workflow of an Oracle client, i.e. basic Oracle instance architecture, connection methods and cursors life cycle. Then I showed demos of several important JDBC features: fetch size; update batching; working with Oracle collections; client- and server-side cursors cache. Examples were focused on the performance of the work with database and its consequences for the client. I’ve finished this section with an explanation how important it is to instrument applications (which is a must for all Oracle-based apps) and a short list of common JDBC driver exceptions. Most of the things I’ve explained are pretty well described in the FAQ, documentation and white papers.

Then I went to database topics, beginning with a general topic of “Performance”. Although I’ve been working in this area for about five years, it was hard to find the right words to begin. And I have to admit it sucked a little bit. Two opening slides & explanation were not the same quality as the rest. I know that, because I didn’t sound convincing to myself. Good thing it was only introduction and it didn’t take too much time to go through. After introduction I’ve reviewed most common Oracle tools for performance diagnostics: out-of-the-box, such as SQL Trace and SQL Monitor, and also some third-party tools, such as MOATS and Snapper. I also showed what scripts and queries I use most of the time in my work. They are very simple BTW; I plan to write about them some day in the blog.
As part of this section I thought I should talk about reading execution plans. But after some thinking I’ve decided to skip this very important and necessary topic and just point to the excellent Christian Antognini’s paper. Instead I showed how to get run-time execution statistics for a plan with an example and where to look at to understand why the query runs slow. Initially I thought I’ll create just one complex example based on a query I’ve seen recently, and then use it throughout the event to also demonstrate several CBO issues such as blind guesses, arithmetic bugs, transformations and something “unknown” which I’ve never seen before. Unfortunately I wasn’t able to build a test case as I wanted, and it served only as an example of a bad performing query.
Then I moved to interpreting Statspack reports. I had sample reports from their production system which experienced some issues. Based on these reports I’ve shown how to approach analysis of the information, what (not) to expect from reports, and where (not) to look for and how to make balanced conclusions about the amount of work done by the database. Also I’ve explained some questionable configuration parameters; where to look next for more data and tried to explain what a potential reason of the problem could be. It is in fact the impossible reason (which I’ve picked up in the OakTable mailing list), but fits perfectly well and explains all issues at once. It is unfortunate system’s DBA missed the event so I couldn’t talk about all of the “why”s. Maybe next time :) I also thought about showing some more reports but once again it took me almost an hour to overview just a single report even without looking at the SQL execution statistics sections. I think I did this part very well and it was enough of analyzing reports for one day.

And the final topic of the day was statistics. This part was a basic explanation of how Oracle deals with execution plans, what kind of features influences its decisions, how they impact building an execution plan and fundamental understanding of what is statistics and what are the things to consider when it should (not) be gathered. I didn’t try to explain the term “cost” in this section and I only talked about cardinality estimates instead. It is enough for beginning, I think. For demos I’ve shown two very common examples when CBO estimates may go off: linear decay for out of range values (which can be “fixed” by manual adjustment to low/high column values) and correlation between columns (which is nicely covered by 11g’s extended statistics).
I’ve also shown in action my script stat.sql which I use all the time to get the basic understanding of table and index statistics.
I have to admit that in this section I only touched this complicated topic. It’s relatively easy to talk about it for the whole day with tons of examples and gotchas. Nevertheless I’m sure that I’ve covered the basics.
And the last part was supposed to be Q&A session (with questions given to me before the event), but I didn’t have time to try and write some code so it went without examples.

All in all it went smoothly; not everything went very well, but some parts were pretty darn good and some were normal. It sucked little bit for 10 minutes or so, which is acceptable. Presentations looked not too sexy – plain text with minimal drawings. Well, it wasn’t about presentations after all. I know there’s plenty of room for improvement in this area. The feedback from the audience was quite positive which is excellent. To me it was a very good experience and I’m happy I did it.

Lessons learned:

  • HDMI may be a problem
  • Don’t try to cover too much the same day
  • Preparation takes longer than expected due to various reasons
  • Preparation brings something new
  • It’s better to know who will attend
  • It would be better not only talk, but also “do something” useful with a real system
  • VM is better choice for demos
  • Google Docs and Dropbox offer very good services

PS. If you think you may need something similar in (preferably) Moscow region, drop me an e-mail. It’s doable.

5 Comments

  1. Posted June 14, 2012 at 05:00 | Permalink | Reply

    Congrats!

  2. Posted June 14, 2012 at 18:35 | Permalink | Reply

    Congrats!Wish you don’t stop and continue these lectures;)

  3. Posted April 2, 2013 at 11:28 | Permalink | Reply

    Link to a short review of the event (in Russian)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 288 other followers